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ABSTRACT 

A project was conducted to identify the major 

conceptual problems associated with secondary cooperative vocational 
education (CVE) programs, develop a model that accurately depicts the 
CVE process, and develop a comprehensive set of measures for quality 
CVE programs. Based on their review of the related literature , the 
researchers concluded that an information system is an integral part 
of the effort to justify and improve CVE programs. They then 
developed a model that characterized the elements of a CVE program in 
terms of inputs, processes, and outcomes .According to the mode? 
inputs arid processes were divided into three categor i es : school 
and linkage. In each category a number of elements were then 
identified as necessary for an effective CVE program. Having refined 
this schema of quality indicators, the researchers then drafted a CVS 
program evaluation model that encompassed the following activities: 
clarification of key CVE program outcomes valued by local leaders, 
establishment of local standards for CVE program outcomes, 
preparation of instruments to obtain outcome arid input indicator 
data, establishment of a system for analysis arid reporting of 
information, analysis of the outcomes of CVE programs by comparing 
program results to outcome standards, arid dissemination 
program information. (MN) 
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Abstract 

The Minnesota Research and Development Center has a major program o f 
research focused on the improvement of secondary cooperative vocational educa- 
tion (CVE ) programs. This research will be divided into several phases; The 
report described herein summarizes the. efforts of Phase i. Phase I goals 
involved: (a) identifying the major conceptual problems with CVE programs, (b) 
establishing a model that accurately depieits the CVE process and (c) developing 
a comprehensive set of measures for quality CVE programs. The strategy used to 
attain these goals was a comprehensive review of the EVE research literature. 

The report is divided into five parts: The first part provides a brief 
historical background for cooperative vocational education, a definition of 
corporative vocational education and a description of the CVE model which pro- 
vided the framework for this review. In part two an attempt has been made to 
address some of the major problems as well as criticisms concerning CVE programs 
and efforts at evaluating them. 

rhu four major issues addressed in part two concern: identifying the proper 
plan for CVE programs in secondary schools, developing a concensus on outcome 
goals for CVE programs, addressing external pressures for accountability and 
evaluating CVE programs. Specific constraints Impacting on each of these tasks 
were identified and a dialogue developed primarily to create a greater awareness 
of the forces affecting CVE programs. This dialogue was riot meant to be defini- 
tive or conclusive and it is hoped that it will serve as a stimulus for further 
di scussi on . 

Part three reviews and summarizes outcomes and outcome measures most commonly 
advocated for evaluating CVE programs. The outcomes reviewed in part three were 
organized into the following categories: economic, educational, social, per- 



sonal and equity. A set ef primary beneficiaries was also identified which 
included: the student, the commun i :y, the employer; the school and the family. 
Outcomes were categorized by major beneficiary to provide an alternative con- 
cept ua 1 mode 1 . 

Part four reviews the major components of a duality CVE program. Using 
the CVE model developed, the resu'ts of research into the quality of CL 
prograjns has been organized sc as to provide a comprehensive but concise 
description of the elements which insure quality in a CVE program. in this 
section the major components of a CVE program are presented in three categories. 
These are school, job and coordination activities. 

A quality program will include in its school component: motivated students 
supportive administration; qualified, competent staff; adequate, modern facili- 
ties; relevant, effective instructional materials; supporting programs and 
planned program evaluations. 

important elements in the job component category include: an effective 
worksite, a relevant and meaningful work assignment, approprate supervision and 
work relationships that include high calibre adult role models. 

Coordination activities in a quality program include: the need for the 
clear assignment and acceptance of responsibility by the employer and instructor 
coordinator for the guidance and direction of assigned student- workers; joint 
evaluation, feedback from interested stakeholders and family involvement. 

The final section describes a procedure to provide information for eval- 
uating and monitoring CVE programs. This information is fed back into the 
system and used to insure that the CI 1 program is meeting its objectives. This 
procedure will be tested in chase two Of the research project. 
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CHAPTER I. 
INTRODUCTION 



Vocational education came into legal exi stance at the Federal level with 
the passage of the Smith-Hughes Act in 1917. Its primary purpose was and con- 
tinues to be the development of an individual ' s specific occupational skills in 
programs of less than the collegiate level;" Individual and societal benefits 
have been the basic rationale for Federal funding. 

Accountability for vocational education dollars has been a perennial issue 
because education for work has been by necessity exper i erital ly oriented: the 
work skills have been taught in educational facilities which are simulations of 
the regular work environment. There has been continuing concern and debate 
about the effectiveness of work environment simulations and the relatively high 
cost of these facilities compared to traditional general education classrooms. 

A solution in a number of schools. has been to adopt the cooperative method 
of educational delivery. This method is simply defined as schools cooperating 
with local businesses who provide students jobs in the regular work situation. 
The dollars saved by not needing to purchase expensive equipment and facilities 
are substantial. Flexibility in the types of occupations students study is 
limited only to the types of cooperating businesses in the school area. There . 
is also the strong logical argument that the on-the-job experience is the best 
learning environment the real world versus the simulation. 

The cooperative method of delivering vocational education had its origin 
prior to the passage of the Smith-Hughes Act. It was mandated as an alternative 
method to school laboratories for vocational agriculture in the regulations for 
the Smith-Hughes Act. It appeared in the subsequent Federal legislation 
regulations for operation of other traditional vocational program areas: With 
the passage of the Vocational Education Amendments of 1968, the term cooperative 



vocational education (CVE) was established as a funding category, and perhaps un- 
fortunately it has been viewed as a program area by some people, not a method or 
del i very approach. 

The Amendments of 1968 carried significant reimbursement incentives for 
development of CVE . The intent was to use CVE to broaden the occupations for 
which vocational education was being provided. State agencies responded in a 
variety of ways ranging from immediate use of federal funds for CVE to slow 
modification of State rules and regulations. The growth of student enrollment 
in CVE was apparently the result of a combination of new offerings and including 
traditional vocational program area students within the CVE category. 

While the positive impacts were significant, a number of questions about 
CVE programs were given limited attention in the expansion effort. As the 
demographic situation changed from the 1970 's tn the 1980 's and a back to the 
basics movement surfaced, many people questioied the release time for students. 
This highlighted a continuing problem associated with the cooperative method of 
delivery-how does one effectively communicate what a CVE program is not. only to 
people directly involved, but also to the tax paying public, ^operational 
definitions for CVE advanced by individuals who are not directly involved in CVE 
programs are significantly different in content. 

Not withstanding the problem of definition and communication of what CVE 
is, legitimate questions of program quality also were raised, e.g. Do students 
learn iiiore on-the-job than in school? Are students developing saleable skills? 
Are students placed in situations that result iii learning appropriate work 
skills? Do instructors/coordinators need all their coordination time? Local 
vocational program evaluations have addressed the cooperative vocational 
education component of programs and provided a range of information for local 
people: While a significant amount of positive publicity and decision making 
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appears to have resulted, limited effort has been made to systematically develop 
a process for gathering data for purposes of analyzing, changing, and 
publicizing the local programs arid for making comparisons between a local 
: program and aggregate data for similar programs within a state. 

In most states, the present evaluation criteria for CVE programs were 
derived from the 1976 Federal Vocational Education Act. These criteria consti- 
tuted an initial effort at establishing a sound basis for monitoring vocational 
education -jr-dgrams. With the advantage of hindsight; it appears that these cri- 
teria are insufficient to measure the many changes which take place in voca- 
tional education programs. This has lead to the call (Evans, 1982; Pratzner & 
Russell, 1983) for new criteria which will effectively evaluate trie full range 
of abilities, skills, and attitudes that are improved by vocational education 
programs . 

Moss and Copa (1982) argued that "students' achievements should a 1 ways be 
measured by the extent to which their relevant capacities have been improved 
(value added)' 1 (p. A). Since all students do not start at the same level, 
fixed program standards are counterproductive. In addition* there is the 
necessity to evaluate educational outcomes other than those directly related to 
economic criteria. Additional evaluation rriteria include growth and personal 
development, social development, and equity outcomes. As Taggart (1983) noted 
"the building blocks of youth programs can only be improved and assured by 
better specification of standards, program content and procedures as well as 
through technical assistance combined with monitoring" (p. 12): 

The research and evaluation issues for CVE programs are generated from the 
complex interaction of sucini, political, and economic forces affecting our 
educational system. The complexity of these forces is at least partially 
responsible for the conflicting research evidence regarding the effectiveness of 
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vocational education programs. Since several major evaluations of vocational 
education have been unable to conclusively demonstrate its effectiveness, one is 
prompted to ask: what accounts for the continued support for the vocational 
education process? 

Srubb (1978) identified several pnssible answers to this question. The 
first answer stems from an implicit American faith in the power of education to 
provide solutions for a wide array of problems. Second is the persistent but 
recurring belief that 'vocational education programs can address such issues as: 
the integration of women, minorities, and immigrants into the work force; 
increased unemployment and technological obsolescence; the inability of tradi- 
tional education to serve the needs of all students; and vast social problems 
such as crime and poverty: The final reason and the one which Grubb (1S78) 
states may be the most powerful is that: "in. a society where work and produc- 
tivity is so crucial to an individual sense of accomplishment, it is immediately 
appealing to turn to work and work experiences as the appropriate way to 
socialize the young" (p. 210). 

'•ihile the goal of this review is to identify the essential elements of a 
quality CVE program, it is not our intention to prove that CVE programs are 
always effective or that they are superior to < , ; other existing work experience 
programs. Rather, it is hoped that our Research and review of the literature 
will improve existing CVE programs by contributing to the attainment Of the 
following objectives: (1) an increased understanding of the relationship be- 
tween critical input factors and targeted outcomes (2) an increased ability to 
control outcomes through the manipulation of specific input factors, (3) an 
increased ability to differentiate external factors from internal factors 
affecting voca,iu.,al education outcomes and (4) an increased ability to monitor, 
measure, evaluate and improve the quality of existing CVE programs. In order to 
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reach these objectives several tasks have been identified. The first task was 
to conduct a comprehensive literature review regarding the measurement and eval- 
uation previously conducted on CVE programs; This review and synthesis is the 
product of this task; 

Definition of Cooperative Vocational Education 

The intention of a cooperative vocational education program (CVE) is "to 

provide students with an opportunity to alternate academic studies with the 

relevant work experience in which they will apply what they have learned in the 

c 1 ass room" (Barton and Fraser, 1978, p. 97). While Barton and Fraser (1978) 

indicated that there were certain definitional problems prohibiting a clear cut 

distinction between work experience and CVE programs, the 1968 Vocational 

Education Amendments specified a definition in which a primary distinction 

' between work experience and CVE programs was that in a CVE program there is a 

definite rel atedness of classroom instruction to "the employment experience. 

Mason and ii lines (1972) gave the following operational definition for 

cooperative occupational experience (COE) programs (COE was broader than CVE in 

terms of the range of occupations students were pursuing). 

Cooperative occupational experience programs have as their 
central purpose the development of occupational competence, using 
employment in a real-life job as a source of. learning. The 
school selects as a training agency a firm. that will provide the 
occupational experiences needed by the student, and the school 
supervises the student's experience. Class work in school pro- 
vides those learnings basic to employment and to the occupation 
sought. The occupational experience is expected to be the source 
for gaining knowledges as well as a vehicle for applying and 
testing what has been learned in the school. (p. 15) 

These definitions explicitly recognize that the primary distinction 

between work study, work experience or part-time work and CVE lies in the 

relationship between the students coursework in school and his/her work site 
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activities. I ho practical implication is that an effort has to no made to 
establish a cooperative relationship between the school and the work site which 
wili » 0 c,t Cl student development and learning. This demands that mechanisms 
exist to coordinate the efforts of the various participants in the CVE process; 
primarily the school, the student, coordinator, ana the employing organization. 
This effort extends beyond simply determining that a student completes 
courses x, y, or- z which relate to job a. 

The coordination effort in the overall context of vocational education is a 
complex educational process which includes such diverse activities as 
administrative functions, teaching, counseling and job placement. The more 
effective the coordination is, the more distinct the CVE program will be from a 
simplework experience program or a part-time job. Barton and Erase. (1973) ate 
correct in noting that the problem of distinguishing between CVE and part-time 
employment is particularly likely to be botherscme if one does not relate the 
process of coordination to expected outcomes. It can be even more difficult to 
make this distinction when classroom activities and/or work activities are 
somewhat si mi 1 ar . 

The criticism that all too often CVE ptv^Vams at the secondary school level 
are nothing more than glorified work experience programs (harm a, in Tonne and 
Nanassy 1970, p. 165) emphasizes the need to recognize the distinctive qualities 
of CVE programs . 

Mason and Haines' (1972) comparison between simple work experience and COE 
programs is classic. They cited the following characteristics as the ei.;ht key 
elements of a CVE program: 

1. The primary and overriding purpose is to provide occupational competence 

at a defined entry level; 

2, The instruction both in-school and at the training station is based upon 
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the student's career goal. 

3. The kind, extent, and sequence of the training station learning 
experiences are correlated closely with the kind, extent, and sequence 
of the in-schoo i learning experiences. 

4. Students may elect the cooperative plan only when they possess the 
employabi lity characteristics acceptable in the market place as well as 
necessary basic knowledges and skills prerequisite to employment. 

5. The employment situation must be truly a training station where the firm 
understands and accepts its teaching responsibility and where a super- 
visor is given the time to act as a training sponsor. •: 

6. The employment conditions are not only legal employment, but acceptable 
by all other standards of the school. 

7. The coordinator has sufficient time to carry out his coordination 
responsibilities and be accountable for quality education. 

8. Instruction is characterized by its individualization, by the use of pro 
jects, by remediation as required," and by interaction with the program. 

Although the 1963 Vocational Education Act refers to CVE as a program, 
recently there has been an emphasis on clarifing this terminology. Thus, a 
proposed 1983 Vocational Education Act (98th Congress, 1st Session, H.R. 4164] 
defines CVE as a "method of instruction in vocational education", as opposed to 
a "program of instruction". This distinction is important since the CVE method 
is a process within and not a substitute for Vocational education. This review 
included vocational education programs in agriculture, marketing, business and 
trade and industry which utilized the CVE method. 

Since the literature has characteristically applied the term CVE program, 
the authors have referred throughout this paper to CVE programs. However, the 
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reader is advised that the term EVE program is meant to indicate a vocation 
education program wherein the EVE method was Used. 

While it is recognized that no definition of EVE will satisfy everyone 
or convey ail of the dynamics involved in a EVE program; the following 
definition is proposed as a useful one for conveying the essence of the EVE 
method: 

Cooperative vocational education is a method of instruction 
within a vocational education program whereby a student receives 
career related knowledge, skills and experience in an actual .... 
employment Setting. The three unique characteristics of the EVE 
method are: 1) class room instruction is directly related to the 
students choosen vocational area and training needs, 2) written 
on-the-job training plans and agreements are cooperatively 
developed by the school and employer, 3) formal supervision and 
student evaluations are a cooperative effort on the part of the 
school arid employer. 



Statement of the Problem 



Despite the evaluation mandate of the 1976 Vocational Education Act and 



its subsequent clarifications, few if any cooperative vocational programs 
have developed an effective evaluation system for monitoring program outcomes. 
There is a lack of both normative and criterion based standards for vocational 
education programs. Without such standards it is impossible to determine if 
a program is meeting federal objectives or even if a program is meeting its 
own objectives. 

Since public education, training, and employment programs are 
important vehicles for helping ycuth-part i cu 1 ar ly minority and 
disadvantaged youth develop needed work related skills, it is 
especially important that a viable and recognized assurance system 
be developed for such programs. If publie programs are not 
considered creditable their graduates will lack credibility, and 
any credentials or recommendation issued by the program will lack 
authority. The real skill gains made by the individuals will not 
be recognized as qual if ications for entry into primary employment 
opportunities. (EEE, 1981, p. 5) 

The goal of the research project of which this review is a part is to 
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develop and demonstrate a management information system for vocational education 
programs utilizing the CVE method of instruction. This system will incorporate 
local input to establish outcomes and to target desired outcome measure levels. 
Instruments and procedures necessary for monitoring overall program quality will 
be developed. As data accrues from the sites where this system will be tested; 
it should eventually become possible to establish appropriate standards and 
outcomes for CVE programs, and to relate these to various levels of program pro- 
cesses and inputs. 

It is intended that this system will also be used in the local program 
formative evaluation and structuring of public information regarding the effec- 
tiveness of CVE programs. A key goal is to be able to demonstrate to school 
officials, local advisory committees and the general public, the advantages of 
the CVE method. 

Organization of the Review 

The model oF CVE Which guided the literature review for this project 
derives from a model of vocational education developed by Copa (1981), (see 
Figure 1). Essentially this model depicts vocational education as an interac- 
tive process in which changes in any component of the model will influence both 
outcomes and fundamental characteristics of other components. 

Copa's (1981) model was originally proposed for planning vocational educa- 
tion but was judged as an appropriate framework for this review since it focuses 
on the interrelationships between a number of variables which must be considered 
in order to realistically portray the CVE process. Efforts to evaluate CVE 
programs must be grounded firmly or, a model of CVE which accurately and real is- • 
tically portrays both the method and the relationship of its various elements. 

The principal components of Copa's model are the individual, the society, 

9 

15 



ERIC 



H 

o 




ERIC 



Copa's Model of Vocational Education. 



the vocational education process, the outcomes generated by the influences on 
the individual and the feedback mechanisms linking the various components. For 
our CVE model, the vocational education process was modified to include both the 
ad'Hfibh of a work setting component and the linkage necessary to provide the 
coordination between the school and work site (See Figure 2). This linkage 
insures that the Work experience component is functionally related to the 
students ongoing educational instruction. Without this relatedness the CVE 
process would not exist. 

The organization of this review is based on the dynamics implicit in the 
Copa (1981) and revised CvE model. Section one (contextual issues) focuses on 
broader social and ideological issues concerning the administration and 
evaluation of CVE programs. The issues examined in this section primarily 
address goal setting and functional problems with evaluating goals. 

The difficulty that education systems have with developing appropriate 
program goals and the standards for measuring these goals may be deduced from a 
brief examination of Figure 2. The society with a work function, the individual 
with a work role and the educational system are in a relationship wherein 
outcome goals may or may not be compatible. In addition, the values of a 
democratic society are often at odds with the equity concerns of individuals. 
Furthermore, the potential for any of these parties to achieve their goals is 
mitigated by the efficiency, distribution and availability of critical resource 
elements . 

The five major subsections in the review of context issues reflect some of 
the tensions evident in our model. Efforts to address these problems as well 
as some alternatives to viewing the problem are considered. The latter concerns 
some cogent arguments against having high school CVE programs. It is the 
viewpoint of the authors that educational reviews should be willing to examine 
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Figure 2. Cooperative Vocational Education 



ERIC 



the arguments of critics as well as proponents. 

Section two, outcome measures , examines literature concerned with the 
methodology and development of appropriate outcome measures for vocational 
education. Although Federal regulations describe a specific set of outcomes for 
vocational education, local vocational education programs hCkv^ continued to 
reach beyond this narrow set of objectives in an effort to achieve goals that - 
are exploratory and beyond the work role. 

In section two, components of the model that will be examined include: 
actual benefits versus potential benefits and the changed individual with a work 
role versus the society with a work function (see Figure 2). Out of these 
dynamics there evolves a pragmatic concern with developing and evaluating 
specific measures for determining the impact of the CVE method vis a vis 
outcomes and changed relationships. The focus in section two is on the 
potential benefits and liabilities associated with the use of certain outcome 
measures . 

Secticn three reviews research related to the inputs, processes, and 
linkage components of our model (see Figure 2). Inputs 3 refer to the human and • 
material resources needed for efficient program operation as well as the time 
and money available for program needs. Process refers to the manner in which 
program inputs are allocated and utilized and to the interaction of the inputs 
within both the school and employment component. The linkage component includes 
all of those elements, resources and plans which are applied in an effort to 
coordinate school and work activities. 

The focus in section three is on examining the relationship of these elements 
to both intended and unintended program outcomes. In addition, various claims 
that certain elements are more conducive to program quality will be investigated 
with respect to ,the available research. 
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Throughout the review, literature which addresses related issues in voca- 
tional education programs without a EVE component as well as those wHh a CVE 
component will be cited; 

The effort has been made to provide a broad coverage of the literature r.bn- 
corning the CVE method. A major risk lies in assuming that programs without a 
CVE component are conceptually similar to programs with a CVE component. This 
risk appears less critical than the risk Of not considering research derived 
from the more substantial vocational education literature. 
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_CHAPTERJI._ 
CONTEXT ISSUES 



Mangum and Walsh (1980) in reviewing the effectiveness of cooperative 
vocational education (CVE) programs stated: 

the bottom line of the evaluations .was a crushing conclusion. 
Though the attitudes of high school work education participants 
we^e much more positive than those of non-participants whil° they 
were in school, two years Tatar, there was relati vely_little dif- 
ference between the two groups in attitudes, current employment 
status, employment stability, wage levels and job satisfaction. 
The participants had enjoyed the coop program but it had 
apparently made little difference in their subsequent employment 
experience, (p. 124) 

A great deal of reiearch has been undertaken with the latent purpose of 
disproving Mangum and Walsh's (1980) findings. However, as long a : the research 



into EVE continues to ignore the underlying social and economics forces 
influencing CVE, Mangum and Walsh's (1980) assertion may remain valid. 
This is true because educational outcomes are severely constrained by the needs 
of the larger society as well a: the overall context in which the program func- 
tions. In fact there are some researchers who feel that this influence is so 
strong that it renders evaluation research worthless, r v 

as a discipline within the applied social sciences, evaluation 
research survives despite mounting evidence that it is frequently 
peripheral and sometimes even damaging to efforts aimed at 
improvements in public policies and programs- (Dunn, Mitroff and 
Deutsch, p. 207, 1981) 

Several major issues must be considered before CVE programs can be validly 
assessed. Some of these issues are reflected in trie following questions: 

1. Does vocational education belong in the secondary school today? 

2. How can acceptable goals for CVE programs be developed? 

3. Does Federal involvement in vocational education programs create sub- 
stantial negative effects? 

4. Can CVE and vocational education programs be effectively evaluated? 

15 

23 



Each of these questions comprises a domain of criticisms and problems which have 
hampered educational programs in general. 

The first question reflects the values of those who believe that vocational 
education has no place in the secondary school, in their view, the work function 
of society would be better served by a different or alternative educational 
model. This question challenges both the legitimacy and role of vocational 
education itself. This is not a naive question since the school is only one 
component in the model (Figure 2) . It may in fact be naive to expect as much 
blit of our school : as some do. 

ntie'-tion two addresses the problem cf defining appropriate goals for CVE 
p.ojrams. Setting goals involves both explicating and prioritizing values 
concerning who are to benefit from the program and what are the benefits that 
can be expected by participants. All too often, actual benefits bear no 
relationship to potential or expected benefits. A thorough understanding of the 
values and expectations of program participants should hej^r to clarify 
differences in goals that may hinder program operations. There is a need for 
expectations to be explicit so that it can be determined if the values we expect 
from our education and programs are both realistic and attainable. 

Quest ion three reviews some of the unintended but nevertheless avers ive 
effects which Federal involvement has had on vocational education drograms. The 
Federal Go Vermont's funding has resulted in the establishment of evaluation 
criteria. These criteria are influenced by the 'democratic role envisioned for 
citizens, by the concerns taxpayers have for the efficient distribution of their 
tax dollars, and by the equity issues generated from both majority and minority 
constituents. Some of the impact of Federal involvement in setting program 
goals will be briefly considered. 

The final question and perhaps the most important one from a research 
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perspective is whether or not it is possible to "effectively evaluate a CVE 
program. \re evaluations worthwhile or are they simply a waste of time and 
money? There are many who believe that the latter proposition is true. Critics 
of evaluation research have poi hted but the futility of attempting to assess 
whether or not a program does make a difference. There is considerable evidence 
that our methodology is still not sufficient to determine cause and effect 
rel at ions hips between school programs and desired social outcomes. 

It is not the purpose of this review to propose solutions to these problems 
nor is this attempted. Rather, these issues are explored because they seem to 
comprise at least one set of contexts which CVE programs must address. Two 
questions that need continuing study are whether or not, these problems actually 
embrace the major issues affecting CVE; and whether or not an attempt to resolve 
them is worth the effort. 

Does Vocational Education Belong in the Secondary Schools Today? 

The first issue examined is the question of whether or not vocational 
education belongs in the high school. Swanson (1982) identified three types of 
arguments that critics of secondary vocational education usually express: 

1. Vocational and/or occupational education is a stigmatizing activity. 

2. Time spent at the secondary level- could be better used than for voca- 
tional education. 

3. An ambiguous argument that for -some unknown reason, vocational educa- 
tion should be delayed until the post-secondary level. 

i 

Reubens (1974) cited several specific criticisms of CVE and vocational edu- 
cational programs. Among these are the following: 

1. The career education movement views young entrants to the labor market 
as full of deficiencies, but it accepts or even ignores unfavorable 
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labor market conditions. 

2. Only a fraction of entry level jobs require specific high school 

vocational training. 

3. Vocational education courses are often offered for some low level jobs 
that do not even require pre-entry school training. 

A. Enrollment in many high school vocational courses is so far in excess 
of the average number of job openings that a regular over-supply would 
result if many errollees did not drop out before graduation or did not 

\ take jobs outside their field of training. 

5. The labor market situation of the young suggests that in. fact there is 
a limited demand for high school graduates and that the type of 
skilled, well payed work extolled by vocational educators is not avai 1- 

o able to most high school graduates, whatever their curriculum. 

: 6 . some courses are preparing students for jobs requiring specific skills 
bi'* often the jobs pay so poorly that students will take unskilled jobs 
with higher pay. 

7. The vocational courses may be so poorly designed that they fail to meet 

industry needs. ) 
8: Although courses are often offered at the high school level, the actual 
jobs which the students are preparing for are only open to those with 
additional education. 
Arguments such as Reubens (1974) are considerably more problematic to the 
CVE process than some of the other criticisms which CVE and vocational education 
programs must face. For this reason, it isjmportant to examine some of the ■ 
assumptions upon Which her arguments are based. The problems which have been 
cited" above can be summarized into two arguments. First, labor market con- 
ditions are often uncontrollable and thus preempt intended outcomes of the voca- 
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tional education system. Second, training is often insufficient, inadequate or 
even useless as it relates to industry needs. 

The above arguments ignore several important facts . First, whether or not 
vocational education is restructured or even taken out of the high school 
setting, the need will still exist to better understand the impact of labor 
market dynamics on training and education program outcomes. Since vocational 
education programs are more job specif ic than general education programs, rapid 
changes in technology and the structure of the labor force impact strongly on 
these programs. The need for relevant vocational programs rests on the ability 
to foresee such changes in- the job market. This is a highly complex problem, one 
which is not going to be solved by vocational education alone. 

Second, Reubens (1974) faults vocational education on its ability and record 
of preparing students for well paying, high career potential jobs. In this she 
ignores the fact that the labor market is segmented into blue collar and white 
co 1 1 ar work. 

Typically, white collar work is more socially prestigious. Social prestige 
is often related to the potential earnings of an occupation or to the number of 
years of education required for entry into the occupation. Talented people will 
most often seek those positions where the financial rewards are greatest and the 
competition for such jobs is keen. 

A college education becomes a mechanism that has both preparation and selec- 
tion functions for many fields. Over time those who cannot compete or who do 
not value the social or economic rewards attached to such highly competitive 
fields $re often forced to pursue jobs that may have less "social desirability". 
Although there are numerous exceptions, frequently such jobs have both lower pay 
and lower career potential. This is also a very complex problem which is not 
going to be solved by vocational education alone. 

19 

27 

ERLC 



Third, the assumption that training is useless because it is offered for 
unskilled jobs or jobs where on-the-job training has been traditional does not 
recognize the fact that in a highly competitive job market (all other factors 
considered equal) an employer usually will select the trained individual par- 
ticularly if he/she perceives that this will result in a cost savings. Thus, 
the person with previous training has a definite advantage. 

• considering the disadvantages which many vocational education students have 
to overcome in the labor market, this is at least one area where they can 
receive assistance. If the vocational education student has some prior 
experience or training in a career field, theoretically this should enhance 
emplo.yability and enable them to compete more efficiently for a job. If all too 
often the jobs that these students are competing for are entry level, low paying 
and require little training, these are often the only jobs that these students 
can obtain. The fact that such jobs lack career potential should be derided 
but' the responsibility rests with the society and not solely with vocational 
education. 

Much of the criticism Which is directed at vocational eduction programs 
stems from the inconclusive and often negative reports (Mannum and Walsh, 
1980) regarding the effectiveness of CVE programs. Such criticism can lead to 
the premature conclusion that if CVE doesn't work then it can just be replaced 
with a more effective and efficient program. This argument not only overlooks 
the cost of other alternatives, which as Welch (1980) mentioned are seldom exa- 
mined, but it also overlooks the fact that it Will probably not be any easier to 
evaluate or determine the costs and benefits of these other programs (Shugoll 
and Helms, 1982). 

The above arguments are not meant to deny that there is a need for the 
consideration of alternative programs, particularly since preoccupation With 
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research into existing educational programs often inhibits efforts at 
investigating other alternatives. Fuller (1982) in an article sensitive to the 
problem of evaluating vocational educational programs stated that n we should 
carefully examine new human development strategies' 1 (page 567); The eentral 
element of Fuller's argument is that it is important for program evaluators to 
acquire broader theoretical understandings so that they can then see beyond the 
confines of the immediate program which they are evaluating. 

Row Gan Acceptable Goals for GVE Programs Be Established? 

Perhaps the most difficult problem facing CVE lies in reconciling 
conflicting outcomes and goals for programs. Local programs are caught between 
trying to meet outcomes mandated by the Federal government and their own reali- 
zation that there must often be a trade-off between goals. Cohen and Solmon 
(1976) noted that "the question of conflicting goals must be reconciled" (p. 
XIX), and a decision made as to who is going to be served by the program (students 
administrators or employers) so that objectives may be matched accordingly. 

The problem with finding acceptable outcome measures for CVE programs is 
compounded by the fact that as Copa and Forsberg (1980) noted "Vocational 
education may have many outcomes, both intended and unintended" (p. 1). 
In an era of increasing cost consciousness, portions of educational programs 
which are not quantifiable in dollars and cents are in jeopardy. 

Hendrickson (1981) pointed out that the strong emphases given to placement 
rates and employment objectives arises from a legislative tiesire that vocational 
educational programs train their participants for labor force participation. 
He indicated that both state and local vocational education directors objected 
to the Federal education requirements because: first, these requirements were 
considered too narrow for vocational education programs and second, job place- 
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ment was viewed as an Unfair standard of accountability for schools; It was 
emphasized that if funding was keyed to the Federal standards it could have a 
detrimental effect tin vocational education programs, since other goals would be 
forfeit in order to meet the Federal guidelines. 

In a democratic system, the plurality of goals seems to present an insur- 
mountable obstacle. The school; as depicted in our model, is literally caught 
in the middle. The challenge becomes one of clarifying goals and of making 
some highly value laden decisions concerning which goals will guide the system 
and which will be ignored. All too often educators bypass this problem by 
assuming that every goal can be met, and thus ignore that certain goals are not 
compatible or attainable. 

Does Federal Involvement in the Vocational Educational Program 
Have Consequential Negative Effects? 

The final report of the National Vocational Education study (1981) stated 

the required statutory evaluations of the placement of vocational 
education students in entry-level jobs related to training has 
led to the; generation and collection of data of dubious validity 
and reliability. Moreover, even if these data were valid and 
reliable, they would have slight utility for the purpose of 
improving programs and deciding on program offerings. (p. 1V/^J 

Numez and Russell (1982) in a study of State legislatures found that 69 

percent of the 209 National Conferences of State Legislature members selected 

for participation, disagreed substantially with having the Federal government 

set goals for vocational education at all levels. McKinney, Gray, and Abram 

(1978), in a study designed to assess the differences in job placement rates 

reported by states (a statutory requirement), found that within the five states 

studied, personnel viewed the data collection primarily as an imposition by the 

Federal government. Furthermore, they were relatively unenthusi asti c about its 

collection since they perceived little possibility that the data would be used 
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for program improvement. 

Since both state and local educators are under a great deal of pressure to 
demonstrate compliance with the Federr.l requirements, evaluation research is 
often blind to other outcomes; Fuller (1982) stated that all too often policy 
issues predetermine the questions asked and the outcomes which will be looked 
for in the evaluation process. Shea (1977) reviewed 200 program evaluations and 
found only 17 that investigated outcomes other than employment and earning 
rates. Raizeh and Rossi (1982) declared that Federal efforts at encouraging 
uniformity of measurement may even prematurely inhibit advances in program 
methodolr jy and that some of the reasons for low quality of evaluations stem 
from the methods used by the Federal government to evaluate educational programs. 

Thus, while there is clear evidence (Numez & Russell, 1982) that there is a 
demand for Federal involvement in evaluating CVE and vocational education 
programs, such involvement would also appear to have a number of undesirable 
consequences which must be resolved. 

Can CVE Programs Be Effectively Evaluated? 

Dunn, Mitroff and Deutsch (1981) stated "an adequate theoretical basis for 
evaluation research is still fundamental ly lacking" (p. 208) and that the 
obsolescence of evaluation research is a result of its preoccupation with 
technique at the expense of examining the concepts and philosophy supporting the 
research, 

Raizeh and Rossi (1981) were also critical of evaluation research but have 
suggested a number of strategies for improving program evaluation in education. 
They noted that "current procedures constrain the quality and the use of eva- 
luation., but how these processes operate is pooriy understood, therefore it is 
difficult to design effective remedies" (page 43). Raizen and Rossi (1981) 
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believe that evaluation research suffers as a proees, because it is being widely 
used without being fully understood. 

t„o major points stand out in the research literature regarding the effec- 
tive evaluation of CVE and vocational education programs. First, the research 
is replete with methodological problems and unexamined biases: In a national 
assessment of cooperative education research in which more than 2,600 documents 
pertaining to CVE and vocational education were reviewed, Cohen and Solmon 
( 1976) identified the following as recurring limitations in vocational educator 

research : 

1. Impartiality of the research was often questionable. 

2. Insufficient samples. 

3. Failure to account for non-response bias. 

4. inadequate validation of survey instruments. 

5. Unwarranted conclusions. 

6. Inadequate information an< Definitions regarding goals, outcomes and 
objecti ves . 

7. Inadequacy of control groups used for comparison. 

One response to some of the problems mentioned above has been cost benefit 
analysis which emerged during the 1976 's; As Shugoll and Helms (198?) noted it 
was often considered a "magic formula". In an examination of some of the short 
comings of cost benefit analysis, they found that: 

the methodological limitations inherent in the technique are too 
great to base such decisions (allocation of funds) solely on the 
resuUs of cost benefits, particularly if the alternative programs 
serve different purposes and have dissimilar outputs, (p. Mi 

The final report of the National Institute of Education (1981) cited 

several other problems with respect to the validity of educational research. 

Among the difficulties noted were the variability among procedures used to 
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secure data unci the lack of comparabil ity between state and local data: tee 
(1979) states ji lack of standardized definitions, the uneven quality of data 
reporting, fragmentation of sources, and the almost complete absence of some 
kinds of data make the entire data base far from adequate as well as difficult 
to use' 1 (p. 73] . The result, as far as evaluation research efforts are 
concerned is to generate confusion and ambiquitj in the decision making process. 
It appears :hat problems regarding the validity of CVE research may stem from 
limitations other than methodological ones and that these are sometimes beyond 
the control of the researcher. 

The second major point concerning the validity of EVE research was very 
well put by Feldman's (1971) comment that "the investigator must navigate be- 
tween the Scyila and Charybdis of hypercri ti calness and Hypocritical ness total 

rejection, or anything approaching it (hypercr iticalness) , may unjustly under- 
mine the research efforts and knowledge of a field" (p. 94). Garcia and Kapes 
(1982) contended that: 

if the researchers applied the strictest research standards to 
the acceptance of any evidence that vocational education had a 
positive effect the likelihood was that no- evidence of 5n effect 
could be found. In this case, the researchers would be condemned 
by their fellow vocational educators for failing to find the 
benefits we all know exist.. On the other hand, should the 
researchers give the benefit of the doubt to methodological 
problems encountered in all of the studies reviewed and conclude 
that many effects of vocational education had been discovered 
they stood to be condemned by the research community for 
accepting "shoddy research methods. (p. 6) 

: Many of the issues discussed lead us back to the question of whether it is 

in fact possible to evaluate a CVE program. Advocating a broader range of 

measureable criteria is not an answer to this problem because, while proposed 

criteria for evaluating the effectiveness of vocational education programs may 

appear deceptively simple, in reality they often turn out to be extremely 

complex and almost impossible to measure. For example the issue of relatedness 
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of training to subsequent employment has necessitated some very sophisticated 
strategies: (Rossmann, 1978). 

the problem of the suspect reliability and validity of a good portion of the 
existing vocational education research also impacts on the question of whether 
and how EVE programs can be evaluated; as was previously noted, at least some of 
the reason for inconclusive research results is beyond the control of the 
researcher. This means that even increasing the standards and quality of eval- 
uation research can not assure that CVE programs will be effectively evaluated. 

The Consortium On Education For Employment (1981) stated "Obtaining quality 
in programs is more of an art than a sci ence . . . the most critical elements for 
judging quality are lost when one concentrates too heavily on easily measurable, 
short term indicators of success" (p. 6). In addition, they noted that many 
indicators used to measure program impaets are poor indicators of long term 
objectives and "Measuring either participant student or program success only in 
terms of the ultimate employment outcomes misses many of the successes achieved 
by individuals within programs and by the programs. . ."(p. 18). 

The Consortium report, while critical of the present state of evaluation in 
educational Federal employment programs listed the following strategies to deal 
with some of the confusion in educational evaluation: 

1. A competency based client centered management system, which will provide 
a basis for relating program inputs to program outcomes and incor- 
porating this information into the planning processes. 

2. Development of adequate assessment instruments for measuring par- 
ticipant gains. 

3. implementation of statewide testing and prof iency standards. 

4. ' An external peer review system. 

The primary shortcoming of the Consortium plan lies in the measurement of 
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' specific outcomes. While "it may be possible to measure certain competencies 

with a proficiency test or some sort of assessment instrument * there are a number 
of outcomes in the affective and social domain which are a great deal more dif- 
ficult to measure (Henri ot, 1972). The task though difficult is not impossible. 

Copa and Salem (1981) conducted a study to find a set of social indicators 
for vocational evaluation in Minnesota. They described social indicators as 
being quantitative in nature and hence measureable. They suggested that while 
the measurement of social indicators is probably in its infancy the following 
criteria are important characteristics of useful indicators: 1) they reflect 
major ends and/or means; 2) they serve' as good proxy measures for several other 
measures; 3) they are easily Understood; 4) they are generally accepted as valid 
and reliable; and 5) they can be feasibly calculated in terms of time, cost and 
expertise available (p. 55). 

Conclusion 

The foregoing questions were reviewed with the intent to better understand 
the diversity and complexity of issues impacting on CVE programs. In the ini- 
tial stages of this review, many such questions presented themselves. As the 
review progressed it became obvious that a process for resolving these questions 
must first understand the interaction between the internal and external elements 
of the CVE model guiding this research. 

; Steers (1976) outlined a method for evaluating effectiveness in organiza- 
tions which may have some relevence for CVE and educational programs in general. 
Steers (1976) contended that effectiveness must be measured by a "process model" 
which will give equal weight to the concepts of goal optimization, systems 
analysis, and emphasis on human behavior. 

Four specific advantages of goal optimization over conventional assessment 
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which Steers (1976) cites are: 

1. The explicit recognition of multiple and even conflicting goals. 

2. The recognition of differential weights for different goals. 

3. The explicit recognition of internal and external constraints limiting 
goal attainment. — — 

4. The allowance for increased flexibility of evaluation criteria. 
Combining the f ram -work outlined above with the systems model developed by 

Copa (1980) for vocational education adds an important dimension with which to 

view CVE programs. If the process and model combined accurately depict the CVE 

~, p — «-* *?> . . 

method, then it may be the start towards a viable solution to the problem of 

program evaluation. 
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CHAPTER HI. 

OUTCOME MEASURES 

This review has already examined some of the issues concerning the develop- 
ment of appropriate outcome measures for CVE programs. This portion of the 
review examines e number of specific outcomes which have been used as Criteria 
in studies measuring the effectiveness of EVE and/or vocational education 
programs. These outcomes are reviewed in five categories: economic, educa- 
tional, personal,' social and equity. These categories were developed by the 
authors after an extensive literature review to identify commonly used outcomes. 
The authors then examined several systems for clustering these outcomes 

(Moss, Smith, Copa, 1972 and Darcy, 1979). Since none of these systems were 

* 

conr.ept.ud 11 y pure it was decided that a simple, if inelegant set of five cate- 
gories would be utilized. It certainly must be admitted that this classifica- 
tion system is not perfect nor does it permit easy discrimination in every 
conceivable case. Its major advantages appear to be that most if not all out- 
comes can be classified in at least one category and the categories are defined 
in relatively distinct and easy to understand terms. 

Economic Outcomes 

The category of economic outcomes had the largest number of measures for 
evaluating CVE pre. ams. The research review found 23 measures which were used 
to reflect various economic outcomes. These are grouped into three, sub- 
categories: wage/earnings, employment, and unemployment. 
toge/Earnings 

Outcome measures in the literature related to wages and earnings included: 

wage rates, gross earnings, average hourly wages, wage on last job, lifetime 

earnings stream data, monthly earnings, earnings on first job, wage rates 

approximately two years later, and wage rates approximately four years later 
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(after graduation): 

Wage rates can be defined as earnings divided by hours worked while ear- 
nings are a gross measure of salary calculated either on a weekly, monthly or 
yearly basis. Each of these two measures offers certain advantages for the 
researcher; Shugol 1 and Helms (1982) stated that "Earnings are a superior 
measure to wage rates, since wages do not account for differences among workers 
in the probability of being unemployed 11 (p. 42). The advantage of wage rates 
lies in their relative ease of computation and collection. It is clearly less 
prcblemmatic to obtain a workers hourly wage rate than his/her life time ear- 
nings stream data. Several researchers have attempted to circumvent the limi- 
tations of the above measures either by combining them or using one or. more 

measures. ; 

Hernstadt, Horowitz and Sum (1979) compared junior and senior high school 

students from several different programs: cooperative vocational, regular 

vocational, work study and general academic They used four measures: average 

hourly wage, beginning hourly wage, final wage earned on last job and the wage 

progression. Stromsdorfer and Fackler (1973) used monthly earnings, ^earnings on 

- . . J >..... 

first job and wage on last job in their analysis of the CVE prbgrani in Dayton, 

Ohio. The rationale for their indicators was that a time earnings profile is a 
more valid measure of program effectiveness even if it indicates only one aspect 
of economic effect. 

Mertens (1981), Meyer and Wise (1980) and Lewis, Glyde, McKee and Kozak 
(1976) used multiple economic measures to evaluate educational outcomes. Meyer 



wage rates four 
National 



and Wise (1980) used wage rates one year after graduation anc 
years after graduation. Their data were taken from the 1972 
Longitudinal Study of High School Seniors. Mertens (1981) reviewed 117 local, 
state and national studies on the effects of participating i/ri vocational 
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education; She found th/t while vocational "education students make more 
initially over time they enjoyed rib significant advantage over general students... 
Her data was based on an analysis of average hourly wages over time. 

Lewis et al. (1976) Used hourly wages on longest job, current job arid first 
job over a two year period, to examine the cost effectiveness of CVE programs at 
the secondary level. They used a random sample of 33 high schools in 15 
districts east of the Mississippi river; Data for CVE students was compared 
with data on students who had non- chool supervised jobs and with students who 
had no jobs. They found no significant differences between the wages of CVE 
students arid the comparison groups. 

Frazier (1981) using wages one year after graduation found average hourly 
wages to be lower for cooperative graduates than for noncooperat i ve graduates 
while Slick and Welch (1974) using weekly earnings (before deductions) and earn- 
ings eighteen months after graduation, found higher salaries for co-op graduates 
/than for the total in-school vocational education graduates. 

| The conflicting results of the studies cited above should not be surprising 
; : S ince most of the data collected suffers from the effects of aggregating all 
I vocational students without respect for curriculum or number of credits obtained 
} in their respective vocational area. This is a problem which has been addressed 
by Campbell, Gardner, Seitz, Chukwamn, Cox and Orth (1981). 

Campbell et al. (1981) divided vocational education students into five pat- 
tern groups based oil their participation in a vocational education program. 
These patterns were operationally derived from the following concepts: 
"Intensity of training, continuity of training, proximity of training to time of 
graduation, diversity cf program areas in which training was received, and the 
addition of logically related study outside the main area of specialization" (p. 
IX). The five categories in assending order of participation were: 
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incidental /personal, explorer, concentrator/explorer, limited concentrator and 
concentrator. Campbell et al. (1981) found that in terms of total effect, women 
and minority women concentrators tended to have significantly higher weekly earnings 
than women with no vocational education. For males, both in terms of total 
and direct effect, concentration was weakly associated with lower weekly earn- 
ings. Looking at hourly earnings, they found that with the exception of minority 
"females (who had a significant positive correlation) and white male con- 
centrators, "participation in vocational education does not appear to have a 
significant impact — positi vely or negatively— on the hourly earnings of most 
employed people" (p. 62). This finding is supported by the research of Grasso 
and Shea (1979); Copa, Irwin and Maurice (1976) and Walsh and Breglio (1976). 
Empl oyment 

The large number of employment measures found was expected because 
employment is a federal ly mandated evaluation criteria. These measures included: 



1. 


Placement rate. 








2. 


Labor force participation rate. 








3. 


Duration of employment. 








4. 


Number of weeks worked. 








.5; 


Hours worked per week. 








6. 


Full versus part time employment. 








7. 


Self employment. 








8; 


Military employment. 








9. 


Number of full time jobs held since 


high 


school . 




10. 


Rates of job advancement. 








11. 


Time needed to find first job after 


high 


school gr 


"aduation 


12. 


Occupational level and status. 









Placement rate is the most common measure used for evaluating CVE-.and vocational 
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education program effectiveness; Despite the apparent quanti tati veness of this 
measure, it is not without limitations. The National Institute of Education 1 s 
(NIE) final report (1981) listed the following conceptual difficulties stemming 
from the use of placement data in program evaluations: 

1. Determining when program completers are employed in "related" 
occupations. 

2. Deciding whether or hot rates "should take into account only those 
individuals looking for full-time employment. 

3. Calculating placement rates. i 

4. Determining the relative importance of placement rates in program 
eval uat ions . 

5. Developing a base against which placement rates can be measured. 
Johnson (1981), McKinney, Franchak, Hal asz-Sal ster , Morrison and McElwain 

(1981), Hernstadt et al. (1979), Frankel (1373) and Cushman, Hill and Miller (1968) 
used rates of employment as one measure for evaluating employment. Frankel (1973) 
reviewed 30 EVE programs and while crediting them for a high rate of job related 
placements also accused them of "creaming", i.e., selecting the most highly 
motivated students. Hernstadt et al. (1979) looked at the employment experien- 
ces of CVt, regular vocational education, work study and academic students 
durihd L.leir junior and senior years in high school "and for/ a follow-up period 
of 16.5 ffldr.uhs to 21 months after graduation. He noted that for the time needed 
to find a first job and labor force participation rates there were no signifi- 
cant advantages for cooperat i ve r educat i on students unless separated by trade. 
Examining specific trades, he found that students from several of the CVE 
programs had a significantly higher rate of participation in the labor force 
then students from the general academie eurri cul urns .' CVE graduates were signi- 
ficantly .''ore likely to work full-time, 35 hours or 1 more per week, then grad- 
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uates from the other three programs. 

NieKinney et al . (1981) examined a number of program input variables and 
their effect on placement rates for secondary vocational education students. 
The six major classes of variables which they used were: • 

1. Employment. 

2. Si zc of community. 

3. Education level. 

4. Extent of community change. 

5. Enrollment composition of the vocational education program. 

6. Services and activities related to the school program. 

They reported that agriculture had higher placement rates in rural areas while 
distributive education had higher placement rates in urban areas. Size of 
community was negatively related to placement rates while ciie median level of 
education in the community was positively related to placement rates. These 
findings illustrate that factors influencing placement rates are often beyond 
the program operators influence. 

Cushrnan et al . (1968) in a study conducted of secondary vocational agricul- 
ture students in off farm occupational specialties selected nine schools 
preparing students for work in ornamental horticulture and seven in agricultural 
mechanization. Following Student graduation, they investigated the number of 
graduates who entered into curriculum related employment. Students in directed 
work experience, Try Out Centers, were compared with students enrolled in a 
similar course of study but without the directed work experience, comparison 
centers. Data Were collected six months after graduation. 

Try Out Center students were found to have obtained a significantly higher 
percentage of job related placements than the comparison center students, 
ft shortcoming of this study was that there was no procedure offered to 

34 

42 



ERIC 



acknowledge how i[ rel atedness" was determined; 

As an alternative to regular employment, several studies acknowledged the 
need to measure self employment as a positive outcome. Woods and Haney (1981) 
reanalyzed the National Longitudinal Studies (NtS) 1972 data sets for self- 
employment rates, as well as labor force participation rates, hours worked per 
week, number of weeks worked during the fiscal year, occupational level and sta- 
tus, full-time employment and rel atedness of work to training and wages earned. 
Comparing vocational education graduates to general program graduates, they noted 
no specific differences in self -empl oyrnent except that there was a tendency for 
blacks to be self-employed less often than whites. The NIE final report (1981) 
also noted no significant differences in self -empl oyrnent between vocational 
curriculum graduates and general curriculum graduates. Using the NLS data chey 
found that approximately five to seven percent of all white male graduates were 
self-employed. However, these data were inconsistent with the labor market 
employment data which showed zero percent males self-employed after four years. 

In examining occupational level and status, Woods and Haney (1981) observed 
a slight tendency for males in trade and industry and business vocational 
programs to be employed in occupations with a higher social economic index\ 
(SEI) than general program graduates. For female graduates, there was a signifi- 
cantly higher percentage of commercial graduates than for general curriculum 
graduates in occupations with a higher SEI. While Woods and Haney (1981) noted 
that this difference did tend to decrease in years; it demonstrates the need to 
differentiate vocational education programs by individual program areas as well 
as by race and sex. In so doing, there is a greater likelihood that positive 
effects will be discovered. 

Several outcome variables, including duration of empl oyrnent , time needed to 
find a first job, rate of job advancement and number of full-time jobs since 
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high school were not as common as placement rates; Marvin, Hartog and Copa 
(1970) selected five schools in Minnesota for a project to demonstrate the 
general feasibility of a system for evaluating local vocational education 
programs and included one question pertaining to job advancement after grad- 
uation in their follow-up survey form, in addition several traditional 
measures of employment success. Molnar, Pesut and Mihalka (1973) examined the 
cost effectiveness of CVE programs versus conventional vocational education 
programs in 12 school districts drawn from Minnesota, North Carolina and, Ohio. 
Using the time needed to find the first full-time job, duration of longest 
full-time employment and the number of different employers after graduation 
as outcome measures, they found no practical differences between graduates of 
CVE programs versus non-CVE programs. 

Slick and Welch (1974) in their evaluation of CVE programs in Pennsylvania 
also used time needed to find the first job after graduation and the number of 
fall -time jobs since high school. They observed that only 24 percent of CVE 
graduates needed more than eight weeks to find their first full-time employment 
as compared to 36 to 40 percent for the conventional vocational education program 
graduates. With respect to the number of full-time jobs since graduating from 
high school, they noted no significant difference between the cooperative and 
conventional education programs. 

It is interesting that both of the programs cited above excluded part-time 
employment as a measure of employment success. Raelin (1981) examined the issue 
of whether or not young workers whose first jobs were part-time lost either sta- 
tus or wages as a result. He used NLS data on young men from 1966 to 1975. His 
conclusion was that "part-time employment is an excellent early labor market 
alternative" (page 321) and that employment policies in this country should more 
throughly examine the idea of part-time employment experiences for youth. 
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in assessing employment related outcome variables, it becomes questionable 
what significance seme of them have for CVE programs; Are the number of jobs 
since high school an accurate measure of success? What constitutes an 
inappropriate amount of time needed to find a first job? How important is job 
longevity? What defines acceptable jODcJdvaneement? Unless the standards for 
evaluating these issues are clarified, the number of valid inferences which 
can be drawn from such data is limited. 
Unemployment 

The three most common outcome measures for unemployment were rate of 
unemployment, number of spells of unemployment during the previous year and 
number of weeks unemployed during the previous year. Rate of unemployment, 
(perhaps the most common indicator) was Used by Woods arid Haney (1981) , Mertens 
(1981), Welch (1980), Meyer and Wise (1980), Hernstadt et al . (1979), Lewis et 
al . (1976), Walsh and Bregl io (1976) , Slick and Welch (1974), and Molnar et al . 
(1973) . Several cf these researchers also examined the number of weeks 
unempl oyed as well as spells of unemployment during the previous year. While 
some of the studies showed that CVE students may initially experience lower 
unemployment; no evidence was found that CVE students experience any long term 
differences in unempl oyment as compared to students from other program areas . 

Harrel 1 and Wirtz (1979) usi ng data from the NlS survey of the Class of 
1972 revi ewed the amount of unempl oyment from 1972 through 1976 for students 
who did not conti nue full-time educati on after hi gh school . As indi cators, 
they used average unempl oyment compensati on rates as well as number of weeks 
unempl oyed each year. Amounts of unempl oyment varied si gnif icantly by race and 
sex. A number of thei r findings support the emphasi s whi ch CVE programs have 
pi aced on work experi ence. Most not abl e were the following: 

1. The more a student worked during the senior year of high school the 
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lower his/her unemployment compensation was during the subsequent 
years. This effect was less pronounced for female graduates than for 
males graduates. 

2. Unemployment rates tended to be persistent and recurring based oh 
the experience of each previous year. 

3. The type of employment was an important predictor of unemployment for 

each subsequent year. 

4. Students who had participated in some type" of job program during high 
school (such as cooperative vocational education or work study) had 
.significantly lower Unemployment rates. 

Since unemployment rates represent the reverse side of employment rates, 
several of the criticisms directed at the use of employment rates as indicators 
of program effectiveness also apply to unemployment rates. Furthermore, during 
recessionary periods, and periods of uneven economic growth, the utility of such 
measures as placement rates and unemployment rates is severely diminished. As 
Walsh and Breglio (1976) noted, 

The relatively low placement rates for programs located in 
Htips with above average unemployment indicates that even 
tho ugh it may not have been difficult to find part-time work 
stations for students. ..it was quite another matter for 
students to find full-time employment after graduation, (p. U) 

Despite the limitations set by economic conditions, employment and unemployment 
rates whe broken down by program area, race, sex, and socio-economic status can 
still yield valuable information. Such information can lead to improvements in 
the areas of educational and equity outcomes. 
Summary 

Twenty-three outcome measures, based on economic data were grouped into 
three subcategories: Wages/earnings, employment, and unemployment. 

The evidence comparing the wages/earnings of CVE students to other students 
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was somewhat i riconc 1 us i ve . The major positive finding appears to be that 
females if; vocational education tend to have sortie advantage in earnings over 
female students in general program areas. 

The employment data consisted of a set of measures of varying relationship. 
The basic concern with this data appeared to be whether or not the student was 
actually employed after high school, a secondary concern was with the quality of 
the employment experience. The evidence here tends to support the importance of 
examining individual fields and not aggregating all programs into one general 
category. In the few studies where this was done, several specific program 
areas appeared to result in greater employment for CVE students as compared to 
general graduates and graduates in other program areas. 

The findings with respect to the other employment measures reviewed 
generally found rib* difference between CVE students and students from other 
curricula. Several of the measures reviewed lacked any type of consistent 
operational definition, thus severely limiting their validity. This was par- 
ticularly true of attempts to look at the quality of the employment experience 
rather than merely whether the student was employed or not. This may account 
for the fact that no study showed any evidence that the rate of job advancement 
was superior for CVE students, or that CVE students enjoyed any long term advan- 
tages in the labor market as a result of their school curriculum. 

The data on unemployment for CVE students supports that in general local 

unemployment rates play a greater role in determining whether or not a CVE 

student will be employed than does high school curriculum. The important point 

that emerges from this research is that the amount of work during high school 

does seem to be subsequently related to lower unemployment rates. Although the 

specific source of employment was an unimportant factor, the evidence supports 

providing either work or job programs for high school youth as a viable means of 
reducing future unemployment . 39 
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Education Outcomes 

Educational outcomes are primarily related to the implicit goals of the 
educational institution itself. The measures most commonly reported were: 
percentage of students graduating, cost comparison within program areas, 
percentage of graduates enrolled in further education, School attendance rates, 
relatedness of training to subsequent employment, and drbpout rates. 

A special word must be said about the conspicuous absence of such criteria 
as grades, basic ski 1 1-s, achievements tests and tests of technical or applied 
knowledge. This review found little evidence of any recent utilization of such 
data as a measure of difference between CVE and Other academic programs. 
Mertens (1981) reported insufficient evidence to measure the effect of voca- 
tional education on the attainment of basic academic skills. Stromsdorfer and 
Fackler (1973) cited several older studies comparing the GPA of CVE students to 
students in other programs. They found no significant evidence to support that 
CVE students had either higher or lower GPA's than students in other curricula. 

in their own research, Stromsdorfer and Fackler (1973) found that CVE 
students had lower GPA's than non-CVE students. But, the CVE student earned 
more credits and spent more time working, thus decreasing the amount of time 
available for study. Furthermore, they caution against the use of GPA as an 
unequivocal measure of success for two reasons: (1) because students may make 
rational trade offs between formal knowledge (reflected in school GPA) and on- 
the-job learning (reflected in performance evaluations), and (2) because as an 
indice of educational effect, GPA's are a measure of intermediate and not final 
program output. They stated that "ideally, we want to measure final outputs" 
(Stromsdorfer and Fackler, 1973, p. 157). 

With respect to research utlizing achievement tests as a comparison basis 
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for EVE and other curricula, again, rib recent national studies were found which 
attempted such a comparison. Stromsdbrf er arid Fackler (1973) cited several 
local studies which did attempt a comparison and found rib strong evidence 
that high school curricula was an important determinant. 

Cushman et al. (1968) compared students in the Try Out Centers (CVE 
program) with students in the regular vocational centers, on an achievement 
test of technical knowledge and competencies. They found that CVE students 
scored significantly higher than students in the conventional vocational educa- 
tion programs. No subsequent studies were found to support or refute Cushman' s 
et al. (1968) findings. 

The lack of utilization of systematic standardized tests to discern 
differences between CVE and regular vocational students reflects the resistance 
of educators in general with standardized performance tests ard summati ve tests 
of academic achievement. Several educators have pointed out that these types 
of tests are costly, time consuming, and quickly outdated. 

Stromsdorf er and Fackler -(1973) and Molnar et al. (1973) reported lower 
high school drop-out rates for CVE enrollees, a fact which the NIE Report (1981) 
has also tentatively accepted. Welch (1980) Using student self-reports from 193 
CVE graduates in Pennsylvania found that 10 percent of those surveyed reported 
that they would have dropped out of high school if a CVE program had not been 
available. Hernstadt et al. (1979) note that 24 percent of CVE students viewed 
the program as having an influence on their decision to remain in school as 
compared to 16.5 percent for students in regular vocational education programs 
and seven percent for those in general programs. 

Mertens (1981) in her review stated that there was insufficient evidence to 
make any conclusions regarding differences; in dropout rates. Cushman et al. 
(1968) and Mangum and Walsh (1980) did not find any evidence that CVE programs 
led to reduced dropout rates. 
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School Atten dance Ra tes 

Lewis et al; (1976) found that students in CVE programs were significantly 
less likely to skip school than students not working. Stromsdorf er and Fackler 
(1973) comparing time absent for senior students in CVE, regular vocational 
education and general programs gave the following respective figures: CVE 
three percent, regular vocational education six percent and general six percent. 

Molnar et al . (1973) looking at the number of days absent, found regular 
vocational education students (x = 7.4 per year) to average less days absent 
than CVE students (x =10.1 per year). Hernstadt et al . (1979) found no signi- 
ficant difference between CVE students and regular vocational education in 
number of days absent during their senior year but found a noticeable difference 
in CVE (x - 18.6 ) versus general curriculum (7 = 29.6). 
Percentage of Studen ts Gradua ting 

One would expect that if the evidence that CVE prevented students from 
dropping out of school was mixed; that trie same pattern would hold for differ- 
ences in the number of students in CVE programs graduating as compared to other 
programs. This proved to be true. Stromsdorf er and Fackler (1973) indicated 
that CVE students had a higher probability of graduating than either regular 
vocational education students or general curriculum students. However, Cushman 
et al. (1968) and Molnar (1973) found no important differences in graduation 
rates for CVE students versus regular vocational education students. 
Cost of CVE Versus Other Programs . 

One important issue which became the subject of some research was whether 
or not CVe' programs cost less per student than other programs. This particular 
line of research proved to be very complex and involved so many variables that 
it was impossible to conclusively determine the overall costs per students in 
various programs. 
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Strbmsddrfer arid FackVer (1973) in a very extensive analysis could not 
determine if CVE cost more* less or the same as other programs: Molnar et al . 
(1973) found rid difference in the cost of providing CVE programs versus regular 
vocational education programs. Lewis et iV. (1976) concluded that overall costs 
per work experience student were approximately 125 dollars more per year than 
for a student in regular vocational education programs. 
Graduates Enrolled in Further Education 

The number of students who go on to further education is ranked as one 
of the most important goals of CVE programs by many secondary school teachers. 
Given the complexities of modern industry many coordinators do not see high 
school as a terminal program but merely as a preparation for more advanced 
training. For students in vocational programs, it is anticipated that further 
education will usually be at a post-secondary vocational technical institute of 
either public or private financing. In some cases* the military might be 
considered as post-secondary training. 

Cushman et al. (1968) was the only study to find positive results for CVE 
programs on this criteria. CVE students demonstrated a significantly higher 
rate of entry into related higher education programs than students in regular 
vocational education programs. Stromsdorf er and Fackler (1973) reported no 
difference among CVE, regular vocational education and general curriculum 
students in the likelihood of acquiring additional vocational education. 

Reporting negative findings, Hernstadt et al. (1979) and Lewis et al . 
(1976) found that fewer CVE students "planned" to pursue further education than 
students in either regular vocational education or general academic areas. 
Whether or not these students kept to their intention is unknown. 

Although there was no consistent evidence that fewer CVE students do pursue 
further education, it seems reasonable to conclude that many students in CVE 
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programs view their curriculum, not. as a departure point, 1 for more training but 
as an entry point for work. j 
The Rclatedness of Training to Subsequent Employment j 

From a logical point of view it seems wort,, .-hi le jtu examine whether or not 
the subsequent employment of CVE students is related (a their training. In the 
first place, one may argue what is the point of an elaborate training program if 
it is not used or does not result in the student obtaining a related job. 
Despite the "logic" in these arguments it has often' proved difficult to 

/ 
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determine what constitutes a "training related placement". In addition, many 

argue that the nature of the job which a student/ obtains is irrelevant since 

../... . .. .. 

there are many extraneous factors involved in job selection. 

Bushman et al. (1968), Lewis et al . (197j>), Stromsdorfer and Fackler (1973) 
and Welch (1980) all found either high lever's of training related placements for 
GVE students or significantly more CVE students in training related jobs than 
students in regular vocational education programs. 

Woods and Haney (1981) and the NIE study (1981) both noted that employment 
in jobs related to training vari ed .corisi dcrab ly from one occupational field to 
another. No attempt was made to/compare the placement rates of GVE students 
against those of students in other curricuTuffis. 

Walsh and Breglio (1976')' surveying urban CVE programs classified them into 
the following categories:/ single or- -ation, general occupation cluster program 

7 % . --- 

and diversified program. 7 The single occupation category qualified as a true CVE 
program since in this /program, student jobs were closely related to their 
classroom training arid occupational area. In the general occupation cluster 
program, jobs were/loosely related within occupational clusters. In the diver- 
sified program, there was no specific occupational training and students were 
placed in a variety of jobs not necessarily related to their major. When data 
/ 44 
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on dropouts arid training related placements were calculated, there was signifi- 
cant differences within CVE groups. Training related placements for the single, 
occupational programs were '42 percent higher than for the diversified programs. 

The average completion rate for the diversified program was between 21 and 
22 percent points lower than the rate for either the single or general occupa- 
tional programs. The fact that each of the programs described classified 
themselves as CVE programs highlights the need for educational researchers to 
carefully define the type of program they are evaluating in order to avoid 
inappropriate generalizations pertaining to program effectiveness. 
Summary 

Most of the data for the six criteria reviewed were either inconclusive or 
mixed. The two strongest trends seem to be that fewer CVE students planned 
further education and that more CVE students obtained training related jobs than 
students in the regular vocational education curriculum. The data suggests that 
many CVE students are not planning to pursue further education and enter their 
CVE curriculum with the intention that it will lead to full-time employment 
immediately after high school. 

Social Outcomes 

The outcome measures included in this category are divided into two sub- 
categories: employer related outcomes and civic outcomes. The employer related 
category includes employer satisfaction with the CVE program and employer 
satisfaction with the Worker. 

A review of employer follow-Up studies conducted by Asche and Vogler (1980) 
found that most employers were asked to rate the performance of the former 
student while the "Vocational Education Data System" (VEDS) asked employers to 
rate the workers training and preparation rather than performance. This review 
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also found that most studies tended to evaluate the worker rather than 
attempting to evaluate the effectiveness or employer satisfaction with the 
school program. 
Employer Related 

Nunez and Russell (1982) randomly selected 2,000 National Association of 
Manufacturing employers to obtain their views about the effectiveness of voca- 
tional education. They found that overall, manufacturers most frequently gave 
secondary vocational education a "c" score (on a scale of a, b, c, d and f), 
with the exception of those companies currently involved with vocational educa- 
tion (ostensibly through a CVE program). Companies involved in vocational edu- 
cation programs graded vocational education higher than companies not involved. 
Lewis et al. (1976) mailed questionnaires to 250 CVE employers. In addition to 
asking them to rate the performance of former CVE students versus regular 
workers, employers Were asked to rank seven possible advantages of participation 
in a CVE program. Of the 68 firms returning questionnaires, most indicated that 
the ability to hire CVE students was the major advantage of participation in a 
CVE program. 

Stromsdorfer and Fackler (1973) selected a sample of 100 employers. Fifty 
of these employers had participated in CVE programs While 50 had not. Forty- 
nine of the CVE employers responded while only 24 of the non-CVE employers 
responded. Stromsdorfer (1973) found that while it was impossible to derive a 
net cost benefit to CVE employers, a significant number of them egreed that CVE 
u programs were ''acceptable to them" (p. 237). 

-Although employer satisfaction with the cooperative student is one outcome 
criteria stated explicitly in the 1976 Vocational Education Act, the NIE report 
(1981) cited several problems with the validity of this criteria. The foremost 
problems are. the frequently low return rate and the built in response bias. 
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Bias iray result for several reasons: CVE employers are often selected by the 
students themselves for follow-up; differences exist in individual standards 
of preparedness for students and finally, employers are unlikely to risk" negative 
dpi hi bhs . 

Asche and Vogler (1980) found that employers preferred graduates of CVE 
programs to graduates of totally in-house programs (i.e., no work experience 
component). Both Asche and Vogler (1986) and Ntirriez and Russell (1982) found 
that employers tended to be quite dissatisfied with the basic skills, reading, 
writing and math of younger workers in EVE as well as non-CVE programs. Frazier 
(1981) compared secondary school EVE students with students in regular voca- 
tional programs using data from the Oklahoma State Department of Vocational and 
Technical Education Student Accounting System. Employer evaluations were 
available only for the years 1978 to 1979, and only , or students employed in 
training related jobs. The number of subjects was 4,668 former students. Based 
on measures of job preparedness, quality of work, technical knowledge and work 
attitudes; employer's rated CVE students higher in every category than regular 
vocational students. ; 

The results of the remaining studies reviewed generally tended to show 
either very high ratings for CVE employees or else rated them higher or equal to 
other groups. In no case were CVE students rated lower in job performance. 
Although Lewis et al . (1976) found that employers rated CVE graduates low on com- 
munication skills, they qualified this by stating that it could possibly be a 
reflection of the low regard among employers for the basic academic skills of 
young workers in general. 
Civic Related 

, In the subcategory titled civic related, two measures were found or 

i suggested as possible program indicators: avoidance of trouble with authority 
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arid citizenship. These indicators are very broad arid imply value based 
judgments which are hard to defend. Nevertheless, if CVE programs are having 
any impact On these broad goals, a researcher must be aware Of them. 

Conflict with authority. Passmore (19S0) analyzed data elements for the 
1977, 1978, and 1979 annual demographic files of the Current Population Survey 
(CPS) to examine the characteristics of jobless youth in the United States. He 
stated that "Juvenile crime and youth unemployment are re 1 ated. . . al though 
whether crime or unemployment are cause or effect or are simultaneously deter- 
mined, is unknown" (p. 3). 

Taggart (1980) found that youth who work, whether in school or out of 
school suffer less unemployment subsequently than youth who do not and that 
there is a statistically significant correlation between youth unemployment and 
youth arrest for a Variety of crime.. Johnson (1981) found a consistent but 
negative relationship between job satisfaction and job performance and reports of 
getting into trouble in his study of four youth groups. Lewis (1976) attempted 
to determine if holding a job while in school had any effect on student disci- 
plinary problems. He noted relatively high levels of disciplinary problems 
both for students who worked as well as students not working. The only measure 
Where he found a difference was for truancy. Students net working were signifi- 
cantly more likely to skip school than students Who were working. 

A study related to the issue of civic indicators was conducted by Free berg 
(1976). His subjects consisted of 184 male and 215 females who had completed a 
worker training program for out of school youth. He surveyed them after gra- 
duation, then approximately six to eight months later. Freeman's purpose was to 
ascertain the re 1 at i onshi p between a number of short-term program completion 
criteria and subsequent post- program criterion measures. Some of the Variables 
in the short-term grouping were quite specific and included such measures as 
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actual police contacts, peer ratings, number of people giving the worker a hard 
time, and family feelings about the trainee; Freeberg's (1976) findings show a 
pattern of significantly consistent correlations between the short term and 
longer term outcomes. His findings suggest that specific criteria can be deve- 
loped which will permit an effective evaluation of he impact of education and 
training urograms on youth socialization. 

Citizenship . Several researchers attempted to assess citizenship through 
such means as voter registration and voting patterns. Stromsdorfer and Fackler 
(1973) looked at voter registration and found no relationship between proba- 
bility of voter registration and types of curriculum. The NIE report (1981) 
also found rid difference in voting behavior between vocational education 
students and general curriculum students. Woods and Haney (1981) suggest that 
both voter registration and voting itself are poor proxies for citizenship. 
Summary 

CVE programs generally appear to receive favorable marks from cooperating 
employers. In several cases EVE students were rated higher than st Jdents in 
regular vocational education programs. It remains unclear which businesses 
benefit most from participation in CVE programs and to what extent CVE programs 
provide an economic benefit for employers. 

The evidence that CVE programs lead to civic benefits, i.e., reduced crime, 
higher voting patterns is far from conclusive and only suggests that working 
while in school may deter some forms of delinquency. Citizenship behavior as 
reflected in voting behavior did not seem to be influenced by type of high 
school curriculum. 
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Personal Outcomes 

For purposes of this review, personal outcomes are broadly defined as any 
outcome which is related to the personal, social, attitudinal, knowledge or 
skill development of the individual. Many of these outcomes are not explicitly 
mandated for evaluation by Federal legislation. However, most educators 
acknowledge the importance of such outcomes in contributing to subsequent labor 
market success. The following measures were most prevalent in the literature: 
job satisfaction, affective work competencies, occupational knowledge, 
satisfaction With school program, basic academic skills, communication skills, 
job seeking skills and occupational competencies. 

Meyer, Crawford and Klaurens (1975) divided personal outcomes into three 
subcategories that provide a measure of "vocational competency." 

I. Technical Competencies : mastery of occupational duties and 
responsi bi 1 iti es . 

- Occupational competencies or skills. 

- Basic academic skills. 

- Job seeking skil Is. 

- Communication skills. 

II. Occupational adjustment competencies: personal adjustment to the work 

environment. 

- uoo satisfaction. 

- Affective work competencies. 

III. Career development competency: understanding ones self and then 
relating to specific career goals. 

- Occupational knowledge. 

- Satisfaction with school programs. 
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Xechnjx-aJ— Competenc i es 

Considering the importance of technical proficiency, it is surprising that 
few researehers have used this measure as an outcome variable. This may reflect 
the difficulties in collecting the data to measure this outcome. In many cases, 
measuring occupational skills would require batteries of tests, most of which 
have not even been developed. The N IE (1981) report stated "there is a major 
practical problem with occupational proficiency measures and criterion referenced 
testing. The first do not yet exist for most occupations and criterion refer- 
enced testing is still in its developmental stage 1 ' (p. IV-20) . Since develop- 
ing these tests would entail a considerable expense, it is no surprise that the 
N IE (1981) study found only 14 states assessing student performance and these 
were described as using tests of questionable reliability and validity. 

Cushman et al . (1968) in their study of CVE students devised multiple 
choice achievement tests to assess "the impact of work experience on vocational 
student acquisition of technical, occupati onal ly relevant knowledge" (p. 28). 
In an ornamental horticultural program, due to the heterogeneni ty of curriculum 
content from school to school they were forced to devise six subtests to use in 
measuring occupational competence. In the agriculture mechanization area, 
program homogeneity allowed the use of a single instrument. Comparing CVE 
students as a group with the regular vocational education students, they found 
that CVE students scored significantly higher on the achievement tests. 

With respecc to basic academic skills, several studies have used grade 
point average (GPA) as a proxy measure. However, no major studies of CVE 
programs were found which directly assessed basic academic skills, job skills or 
communication ability. Some evidence supports the importance of assessing these 
abilities. 

The Research for Better Schools Study (RBS, Richards 1981) of regional 
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employers in Delaware, New Jersey and Eastern Pennsylvania used a questionnaire 
to survey 178 employers on the importance of 9 transition skills (school to 
work). The questionnaire included basic academic skills and communication 
skills; The categories were derived from 11 earlier studies of employer atti- 
tudes towards young workers. The RBS survey found communication Skills to rank 
number three in importance to employers while basic academic Skills ranked 
fourth in importance to employers. The N IE (1981) report also listed basic 
academic skills, as well as ability to cope with job change as major goals for 
vocational education. It suggested that these goals are implicit in the Federal 
vocational education legislation. 
Occupati onal Adjustment Competencies 

The two outcome criteria in this category are affective work competencies 
and job satisfaction. Beach and Kazan as (1981) defined affective work competen- 
cies as "social psychological characteristics including habits which are mani- 
festations of preferable work behaviors that persons demonstrate" (p. 51). 
Their research indicated that proper work habits are critical for entering the 
job market as well as maintaining a successful employment record. They noted 
that despite the evidence supporting the importance of affective work competen- 
cies, it is an area which is often overlooked by many educators. 

A substantial amount of research attempted to evaluate outcomes chat could 
be classified under Beach and Kazanas's (1981) definition. The only personal 
outcome category which was found more frequently in the literature was job 
satisfaction. Perhaps a reason for the large amount of research In this a^ea is 
that it is an extremely broad classification. In Beach and Kazanas's (1981) AWCI 
inventory, it Included 59 individual characteristics grouped into 15 clusters. 

Luft and Suzuki (1981) described affective characteristics as non-technical 
employment competencies and identified 59 non-technical competencies from which 
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a list consisting of 31 items was developed. This list was sent to coordinators, 
counselors and employers who had worked with CVE students. They were asked to 
indicate the level of importance of each competency on "a five point scale. Tne 
results showed that coordinators, counselors arid employers tended to agree on 
the importance of these competencies. The similarities in the competencies 
identified by Luft and Suzuki (1981) and those in the Kazjnas inventory (1981) 
are striking. 

Two studies of CVE programs which used selected affective work competencies 
as outcome criteria (Slick and Welch, 1974 p.nd Lewis et al . 1976) found no 
difference between CVE students and non-CVE students. Johnson (1981) found that 
CVE students scored lower on work attitudes and ability to get along with co- 
workers than regular vocational education students. 

One of the most ambitious studies in the area of affective competencies was 
• conducted by Dunn, Ridley, and Walker (1982). Their study used regular voca- 
tional education students as subjects and drew on 25 of the 51 secondary level 
educational agencies served by the Office of Occupation and Continuing Education 
in New York to collect data on 7,«lb students including 3,842 graduating seniors 
enrolled in 75 different occupational areas. They compared the students from 
occupational high schools with students from home feeder schools on nine atti- 
tude scales which included a broad grouping of affective competencies as outcome 
variables. Two of the scales used in this study could not be categorized 
strictly in terms of affective competencies since they also measured attitudes 
toward school as well as job finding competency. 

A major purpose of the Dunn et al . (1982) project was to develop a 
reliable and valid instrument for measuring students* sehool and related social 
attitudes as well as to compare the attitude of students in occupational 
programs with students in other high school programs. They found that occupa- 
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tional seniors clearly exceeded home school seniors in four of the nine measures 
used: 1) positiveness towards their school experience, 2) positiveness toward 
their classmates, 3) confidence in their career goals, and 4) confidence in 
their job keeping skills. 

In examining the results for post high school graduation job satisfaction, 
eight EVE studies used job satisfaction as a criteria while one study compared 
students satisfaction with in- school work education jobs to student satisfaction 
with a part-time job held while attending high school. Of the eight studies 
reviewing post-high school job satisfaction, six found little cr no difference 
between CVE students and non-CVE students. Stromsdorfer and Fackler (197;) 
found CVE students to have a higher deq~ee of job satisfaction than non-CVE 
students and Welch (1980) who did not use a comparison group, reports that 66 
percent of CVE students were satisfied with their jobs. 

Snberman (1974) compared data from 1,016 students in 50 work education pro- 
grams with data from 696 students no^ in a formal woi k education program but who 
were holding part-time jobs. Students in formal work education programs were sig- 
nificantly more satisfied with their jobs than students not participating in a 
structured program. Silverman (1974) suggested that students participating in a 
school supervised work education program achieve greater satisfaction because of 
the meaningf ulness attached to a job which is closely related to their career goal 
Career Development Competencies 

Two outcome criteria are listed in this category: occupational knowledge 
and satisfaction with the school program. Occupational knowledge can be defined 
as an individuals general knowledge of the skills, requirements, duties, and 
responsibilities in a variety of vocational areas. This is an important 
category in many states. For example, one of the explicit goals of secondary 
vocational education in Minnesota is the exploration of occupations with the 
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intention of providing the student with information needed to make realistic 
career goals, (Minnesota Rules and Regulations, Department of Education SMCAR 
1; 000811) ; Edin (1979) in a study of 155 teachers in 19 secondary vocational 
centers in Minnesota asked them to rate three goals in terms of importance: 
occupational exploration, preparation for further training and job placement. 
Sixty six percent rated occupational exploration as the most important goal. 

Notwithstanding, the obvious importance attached to the goal of occupational 
exploration j this review found only one study which attempted to measure CVE 
programs for effectiveness on this variable. Lewis (1976) et al. gave a short test 
consisting of 17 items to current CVE students. Each item included a job 
title and three jot> descriptions, one of which was a match for the job title. 
When students holding school supervised jobs were compared with students 
holding non-supervised part-time jobs and students not working, they 
found that male job holders scored significantly higher than hah job holders. 
However, the difference in scores, approximately one percentage point is pro- 
bably of no practical significance. 

Examining students' satisfaction with their school programs was a par- 
ticularly popular effort for EVE researchers. Six studies used this criteria as 
an outcome measure and all indicated a consistently High level of satisfaction 
on the part of students enrolled in CVE programs. Johnson (1981) and Welch 
(1980) both noted a very high level of program satisfaction on the part of CVE 
students, while Hernstadt et al. (1979), Lewis et al I. (1975), Slick and Welch 
(1974), and Frankel (1973) each concluded that students in CVE programs were 
more satisfied with their school programs than were students in a variety of 
other comparison groups, including work study programs, regular vocational edu- 
cation programs and general education curri cul urns . 
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Summary • 

In looking across the studies reporting on the job satisfaction of CVE 
students versus those in other programs, there appears to be no strong evidence 
that CVE students are any more satisfied with their jobs than are other students. 
In the development of affective work competencies, the evidence was mixed. One 
study reported a negative effect for CVE students; one a primarily positive 
effect based on self-reports; and others found no difference in either self- 
esteem or relationship skills between CVE students and students in other, 
programs. Occupational knowledge was a criteria which only one study attempted 
to measure and this study did not find many practical differences between CVE 
students and the working students. No research was found that CVE programs 
result in the development of greater occupational skills, job skills or other 
kinds of abilities categorized as technical competencies. This finding pri- 
marily reflects the lack of attention directed to this area as well as the dif- 
ficulty of measuring such skills. 

The most positive findings for CVE programs was in the area of overall stu- 
dent satisfaction with their school programs. All six studies reviewed found 
CVE students to be either more satisfied or significantly more satisfied with 
their program than students in selected comparison groups. 

Although personal outcomes are consistently viewed by educators as a pri- 
mary goal of schooling, it appears that standardized, systematic efforts to 
measure such outcomes are still only in the development stage. The task remains 
for educators to develop effective mea>ures of thdse outcomes. 

Equity and Equal Opportunity Outcomes 

The philosophical arguments over what equity and equal opportunity mean 
would extend well beyond the scope of this review. The fact that the defini- 
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tional problems seem-to be far from settled (Biirbules, Lord and Sherman 1982) 
presents a dilemma for educators since evaluating the results of the services 
provided for special populations is an outcome specified in the Federal regula- 
tions for evaluation. The five criteria in this category are: (1) services to 
women, (2) services to members of minority groups, (3) services to handicapped 
persons, (4) services to disabled persons and (5) services to persons of limited- 
English speaking ability. 

Hendrickson (1981) noted that less attention has .been given to evaluating 
these requi rements than for any other federal requirement. The NIE report (1981) 
stated that some of the reason for this may be do to inadequate program 
resources as well as i nadequate Federal guidel ines provided for program deli- 
verers. The NIE ( 1981) report noted that for a state to evaluate services to 
spec i al needs populations in terms of planning and operational processes as .we li- 
as avaiiabiiit^of services and access to programs would call for a "battery of 
sophisticated and costly evaluations" (page IV-XXI) . 

There are others who see a different reason for the 1 ow emphasis on eva- 
luating services to special populations. Barton and Frazier ( 1980) stated that 
schools tend to encourage destruct i ve societal stereotyping through restrictive 
course offerings, curriculum material's* and biases on the part of teachers, 
counselors, and administrators . They noted that vocational school programs in 
parti cul ar have tended to restri ct the role of women in the 1 abor market. 

Reubens (1974) echoes Barton and fnzer (1980) in her statement that " A 
significant aspect of' vocational courses is their sex role stereotyping and par- 
ti cul ar ly, the relegati on of girls to the occupations that lead to 1 ess 
favorable 1 abor market outcomes or have no connection to work at al 1 " (p. 301) . 
McClure (1979) stated that "Vocational schools prepare minorites and women for 
their place in the economic and social order" (p. 284). McClure £1979) .implied 
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that this is as true today as it was in the last century. 

The project on National Vocational Education Resources of the National 
Institute at Berkeley conducted a random survey of 1,200 local agencies pro- 
viding vocational education in ten states during 1980 to 1981. Benson (1982) 
compared the percentage of districts hiring or reassigning staff to promote sex 
equity between the years 1978 to 1980. Only five percent of the secondary 
district programs as compared to 20 percent of the postsecondary districts 
reported hiring or reassigning staff to promote sex equity; 

Benson (1982) also used this data to compare the percentages of secondary 
and post-secondary agencies incurring excess costs for handicapped and 
disadvantaged students. In five service areas, the percentage of agencies 
incurring special costs at the secondary level ranged from four to 23 percent 
while at the post- secondary level, the range was 20 to 42 percent. Hamilton 
(1979) reported that 60 percent of the handicapped people between the ages of 16 
and 64 do not finish high school and that few are acquiring the skills and 
training necessary to prepare them for employment. The research suggests that 
vocational education may be ignoring the. intent of the Federal equity 
requirements . 

Another explanation for the low priority assigned to equity issues stems 
from the conflicting goals which confront CVE and vocational education programs. 
Lecht (1974) suggested that economic and efficiency criteria can sometimes 
conflict with equity concerns. He noted somewhat facetiously that if effec-. 
tiveness and efficiency are to be the primary goals for vocational education 
than it would make sense to invest resources in the most capable people and to 
ignore the special needs population. Numez and Russell (1982) in their survey o 
manufacturers found that they assigned a low priority to establishing special 
opportunities for minorities and for providing nontraditionai occupational 
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training by sex. Somewhat paradbxicly the NIE (1981) report showed that while 
most State vocational education directors support the federal . evaluation 
requirements for special needs population^ only brie State favored using "serving 
special needs" as an indicator of program quality. 

Looking more specifically at research concerned with CVE programs, Walsh 
arid Breglid's (1976) study on urban CVE programs indicated that since 1968 there 
has been a substantial increase in the number of programs for disadvantaged and 
special needs students. Nevertheless, with respect to outcomes for minorities, 
they round that the range of occupations was narrower for minorities than non- 
minorities and that high school minorities were slightly less satisfied with 
their cooperative experience than their non-minority counterparts. Consi der i ng 
the outcomes for women, Walsh and Breglio (1978) found that they were lower than 
for men. Women had a more restricted occupational range and when they entered 
the labor market, their wages were significantly lower than the wages that men 
received "even when both had been trained in the same occupational area" (p. 35). 

Molnar et al . (1973) found that on the average, CVE programs handled 
proportionally more disadvantaged students than noncooperat i ve programs in the 
junior year, but that there was rib difference in the senior years for proportion 
of disadvantaged students served by programs. The percentage of handicapped 
students in all programs, cooperative as well as non-cooperative was low. Also, 
there was no difference found uy type of training. They noted that more CVE 
graduates were females. Non-cooperative graduates tended to be male and the 
percentage of graduates who were non-white was greater for cooperative programs 
than for non-cooperative programs. 

Lloyd (1981) used a panel of experts to select five states for a study of 
quality indicators for CVE programs. State directors in these five states recom- 
mended 80 high schools of which 67 were included in the study. One hundred 
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thirty- four EVE students, 126 training sponsors and 75 teacher coordinators 
responded to questionnaires and survey instruments. 'Lloyd found that 73 percent 
of the teachers had special programs for special needs students including han- 
dicapped and 'disadvantaged students. Forty-nine percent had programs for high 
school dropouts. The percentage of .disadvantaged students enrolled in the CVE 
programs ranged from ? low of four percent to a high of 16 percent within the 
five states. The mean for all five states was eight percent. The percentage 
of handicapped students served ranged from six tenths of a percent to three 
percent with a mean of one percent. A rather surprising finding was that 14 of 
the 75 teacher coordinators dealt With all of the handicapped students. With 
respect to sex equity, Lloyd found that the majority of males and females were 
primarily enrolled in the traditional male and female occupations. Ethnic 
groups appeared to be adequately represented in the study group; however, some 
confusion with respect to the number of responses on this issue rendered the 
- i rid i rigs suspect. 
Summ ary 

Although there has been some significant gains made in services to women 
and minorities over th„ last decade; the evidence suggests that the work to 
establish an equitable distribution of services and opportunities to women, 
disadvantaged, minorities and special needs students is far from completed. 
More troubling !s the -fact that the evidence shows a mixed commitment to the 
goals Of equity in education. 

CVE programs by definition are dependent upon local employers for dobs and 
training stations. Perhaps because of this fact they appear to reflect the 
current equity status of the World Of work. 
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Conclusion 

A number cf outcome measures were examined which have been used by those 
assessing vocational education programs. These measures are assumed to reflect 
what are both implicit and explicit goals for Vocational education and 6VE 
programs. None of the theoretical frameworks reviewed provided a system wherein 
goals, criteria and outcome measures were easily distinguished and conceptually 

distinct. * 

The literature reviewed suggested the two major limitations of criterion 
outcome measures stemmed from: 1) philosophical differences concerning the 
goals of vocational education and 2) measurement problems within a dynamic 
social system. A major philosophical issue contributing to some of the goal 
conflict in vocational education derives from the lack of a concensus concerning 
the purpose of vocational education in the public secondary school system. 

This latter issue highlights the need for careful and continuing examin- 
ation of the goals established for vocational education programs. Without 
a concensus on goals, decisions cannot be effectively made regarding the 
allocation of limited program resources and manpower, nor can appropriate 
program outcomes be selected and evaluated. 

The outcomes reviewed derive from a social system which includes a shared 
culture, cultural values, roles, role expectations, and institutions or groups 
wherein members conform to shared standards of behavior (Moss, Smith, Copa, 
1972). Every society is an organized networK of smaller groups. The culture of 
each group providing a way of life shared by its members. Groups and institu- 
tions connote roles (some assigned and some voluntary) in which members are 
expected to behave in prescribed ways . These group expectations induce a set 
of standardized behaviors which denote "expected roles' 1 . Both groups and 

individuals share an expectation of derived benefits from this organized 
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coll abbrat ibh (Moss* Smithy Copa, 1972). 

Benefits are assessed by measuring outcomes which should reflect the 
perceived goals and mission of each group, in a socio-educational nexus, goals 
and outcomes , are intertwined so that outcomes are not always distinguishable 
fioiii goals. Outcomes may be categorized within a schema which portrays various 
aspects of the individuals r*ole - ; <s impacted by the demands of influence groups 
or stakeholders in the larger social structure. In our CVE model these include, 
the individual, the family, the i;-:nbSl, t.ie "employer and the immediate 
community. The individual i s u -hi ly viewed as the hub of p-z\\ z 7 tb^Gc 
subsystems. Figure 3 illustrate, these relationships. 

The Figure 3 social system model suggests that five ^'eLS of goals can be 
derived from an analysis of the rJle expectations illicit in- the individual's 
perfon,, nee within each of the constituent groups in the i ramework. Thus, there 
would be a set of goals for the individual - within a school context, within a 
>work context, within a family context, within a community content and a more 
general set of goals deriving from the individuals perspective of him/herself, 
in general, one would expect that outcome criteria or benefit indicators could 
be developed which would adequately measure the extent of goal attainment. 

In the review of literature, the outcomes found corresponded to the 
expected roles and behaviors prescribed for the individual in the conu.xt of the 
CVE program. 

Figure 4 presents the outcomes reviewed, abstracted and "organized by 
beneficiary categoric;. The argument can be made that all of society benefits 
from the education of the individual, and that several outcomes cut across 
certain classifications. However, it is maintained that outcomes r in be viewed 
as primary (direct) and secondary (indirect) benefits. The effort in Figure s 
was to categorize items by assigning outcomes to the primary beneficiary groups 
reflected in the literature reviewed; 
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Figure 4 . CVE Social System Model with Benefits, 



64 



Figure 4 reflects outcomes which have been studied. It does hot endorse 
values assigned to outcomes , although" it is hoped that outcomes with a higher 
social priority are indeed those that are being studied. The lack of studies 
addressing outcomes pertaining to family benefits and conversely to the families 
impact on the CVE program may be an issue that has to be addressed. 

If local goals for CVE programs in secondary school are to be established, 
it is important that local community value priorities underlying these goals 
be examined. This should be done in concert with an examination of the values 
priorities underlying the goals set for CVE programs by both Federal and State 
legisl at ion: 
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CHAPTER IV. 

INPUTS AND PRQeESSES 

This section of the review examines research literature pertaining to 
the inputs and process components of the CVE model proposed. The literature was 
reviewed to find those program characteristics which had some, empirical basis 
for being considered as an indicator of program success. Such indicators 
should have the ability to distinguish a good CVE program from a fair CVE 
program. characteristics referred to in this section as quality indicators 

are divided into the following categories: school components, job components, 
and coordination activities. 

Referring again to bur revised model of vocational education (Figure 2), 
the school components and employment component consist of both inputs and pro- 
cesses. School inputs includes such items as students, staff, textbooks, 
classrooms, materials* payroll* budget, and any other resources which in 
themselves could not be construed as a process. Similarly employment inputs 
denote the job site, supervisors, equipment, payroll and all other resources 
used \ r t the training of the student. 

Scho 1 processes consists of any systematic series of actions directed 
towards the attainment of one or more program goals. A number of these outcomes 
were described in the previous section. These activities could be placed in a 
heirarchy, from the most important to the least important. One of the problems 
with this heirarchy would be that often a higher rated process could not be 
implemented were it not for the presence of a lower rated process. For this 
reason no such scaling was attempted, although it see^s reasonable to assume 
tha*. some processes are more important than others. 

Employment processes includes training activities, work activities and 
supervisory activities. The major difference between employment processes ->nd 
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school processes is that the primary focus of the" school processes is the indi- 
vidual while in the employment process the individual is secondary to the goals 
of productivity and profitability. The school exists to serve the individual 
while the worker exists to serve the company; This is a consideration that many 
individuals who are not in the private sector ignore. 

The coordination between the school component and the employment component 
takes place through a series of formal and informal activities. Some of these 
activities include visits between the coordinator and employer, employer input 
via advisory councils into the curriculum and joint evaluations of both students 
and program: . 

Several caveat:; should be mentioned regarding the indicators in this 
review. First, the list that was compiled is by no , --ans exhaustive. Second * 
the justification for an indicator's inclusion is based on research findings 
which differed considerably in scope and quality. Third, the research does not 
enable as to determine Which are the critical indicators and which are essential 
but not critical. In theory, this -could be accomplished if a common measure of 
significance existed which would permit weighing the value of one indicator 
against another. 

Finally, considerable latitude was taker, i-i restating some of the charac- 
teristics used as quality indicators in this study. Whenever it seemed apparent 
that two or more characteristics Sere similar in nature or were describing simi- 
lar processes, they were combined. Host characteristics are not phrased 
exactly as in their original study. It is hoped that the descriptions are not 
so broad that the essence of the original characteristic is lost. Since there 
was no body of •terature defining key quality indicators, the researchers had 
to judge which indicators merited special consideration due to their ability to 
positively influence program . jtcomes . 
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This portion of the review is characterized by two important features. 
First, each of the studies reviewed attempted to make an association between a 
program input or process and some objective measure of program success, i.e., 
a program outcome. Second, the literature considered included studies drawn 
from research on youth employment and training programs, youth work experience 
programs, experiental education programs ard CVE programs. Since the outcomes 
of these programs are similar, it was reasoned that factors which distinguished 
good programs would be general izeahle across program areas . This is an assump- 
tion whi ch may be chal lenged. 

School C mpo^ents 

This section of the inputs and processes includes the many diverse elements 
of the educational process itself (see Table 1) . A great deal of CVE research 
has been directed at examining the impact of these elements on student goals and 
moti vaticr.. Given the American educational phi iosophy, it is usually the school 
which receives the credit when these goals are met and the blame when they ere 
not. Thi s situation exists despite the fact that the school is but a partner in 
the eaucationa.1 process and in a CVE program, industry and the economy are 
additiona 1 partners. Nevertheless, it seems likely that the school will continue 
to carry the ma: * r share of re^oonsibi 1 ity for the success or f ai lure of ruch 
proyrams as CVE . 
Students 

T ne basic cr iter i a for enrol 1 ing students in CVE programs has been that 
they are motivated and can henef it from the program. Many schools, ' 3wever , 
place a grade level and age ( LI oyd , 1976) requirement on entry into the CVE 
program. With respect to "motivation" (a rather difficult qual ity to measure) , 
it appears that many coordinators attempt to control this by working coc-ely 
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Table 1 



QUALITY INDICATORS FDR A COOPERATIVE VOCVIONAL EDUCATION PROGRAM: 

SCHOOL COMP ONE Mi- 



Students 



1. programs should be balanced in enrollment with respect to age, sex, race, 
and socio-economic status. - . 

2. Students selected for the program should be highly motivated. 



15 
16 



Admin i strati on 



3. The administration is responsive to the needs of the program, individuals 
and businesses connected with the program. 

4. The administration takes responsibility for program placement. 

Staff 

5 School staff should resemble racial balance of the community; 

6*. Teacher-coordinator is provided with extended time when and if needed. 

7. Teacher-coot': inator teaches the related class. 

8 Flexible salary scales for vocational education teachers. 

9* Appfupriatp coordinator and student ratio and coordinator work load. 
10* All si iff have valid teaching licenses in their specific vocational area, 
ll! Each staff member meets the vocational education 1 work experience require- 
ments of the state. . ...... ., , 

12 The coordinator is implementing the gi.als „f the total educational program. 
13! Staff members obtain help from other teachers in recruiting and selecting 
studentf. 

14. Coordinator has a rapport with youth. 



Faci 1 ities 

A private area exists for tne coordinator to eounsel with students. 
Adequate space, equipment and instructional materials are provided for the 
coordinator to carry out their responsibi 1 ities 

17. IransDortatiOn to jobs is available. _ _ _ . 

IS. The cooperative staff has adequate control of facility and equipment 
utilization. 

Instruction and Training Materials 

19. Class schedules allow students flexibility to participate in directed work 

experience. _ . — . 
20 Reading level of materials r.acc.'ies student's reading level. . 

21. Competencies, objectives and nu.ber 0? hours 9i instruction are appropriate 
for the learning of the specific occupations. 

22. Instruction and training materials are current and meet prevailing business 

needs . — . — . . 

23 Students are provided with meaningful laboratory and project experiences 

which allow them to apply knowledge and skills learned in the classroom. 
24. Competencies exist to insure acquisition of basic education ski!-s 
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Table 1 (continued) 

Instruction and Training Materials (continued) 

25. Instruction and training in affective work competencies is available. 
26; Instruction and training in career exploration and decision making is 
avai Table; 

27. Student youth organizations are available and utilized. 

Speci al Programs 

28. Counsel i ng services are available. 

29. Job placement serv ' : are available. 

Program Planning and Evaluation 

30. A comprehensive training plan is used to direct the achievement of learning 
experiences and to determine 1 earning performance. 

31. Students are kept informed of their ski lis and knowledge achievement via a 
comprehensive training plan directed by t!'.-e cooperative coordinator. 

^Indicates an item where there is some disagreement in the literature 

with school counselors or other teachers who will refer "motivated" students 
for CVE enrollments (Lloyd, 1976). 

A few researchers suggested that there should be stricter entry require- 
ments for CVE students. McKinney et al . (1981) suggested that student admission 
to CVE programs be restricted to students with interest and potential the 
traditional vocational education legislation positions. Mangum and Garth (1978) 
advocated restricting the numbers of "unmotivated youth " placed in work 
exper i enced programs . 

Taking a stand quite opposed tri stricter program entry requirements, 
(Fraiikelj 197 r ; condemned CVZ programs for their Use of "proper student 
attitude" as a selection criteria. F: ankel (1973) concluded that program coor- 
dinators should not exclude students on the basi'i of attitudes (since these are 
often discriminatory) apd that guide! incs should be establ ishsd for CVE programs 
with respect to their exclusion of students for either behavioral or attitudinal 
reasons. 
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the issue of whether or not there should be special entrance qualifications 
for CVE programs reflects a basic difference in educational philosophy regarding 
both the role of a CVE program as well as its potential ability to deal with a 
heterogeneous population. , 
Admin i st rati on 

Several studies emphasized the need for the school administration to be 
-es possible for the over -ill needs of the CVE program. Benson (1982) stated that 
"The s-cond factor which dfe terries qa«1 -tj in vocational education is admini- 
strative leadership 11 w ^pMed -.hat vocational education has often 
been placed in the role of the ujV steochild by principals who have had little 
interest c~ background in the trades or private business. Lloyd (1976) found 
that 90 percent of the teachers in hi: study believed that a positive rela- 
tionship :ith the school administration was vital to coordinating a successful 
CVE program. 

wubbena's (1986) finding, also indicated the need fo, an administration which 
ecu assume a supportive role in the operation of a CVE program. Wubbena U980) 
conducted an investigation on the perceived importance of 288 selected program 
characteristics. The population of his study consisted b* all secondary distri- 
butive education teachers within the central region (13 states). From this 
ponu^rion, a random sample of 450 teachers was selected. To avoid any possi- 
bility of bias, these 450 teachers were further divided into 6 subgroups. Three 
groups were Used to rate the import .nee of each item, while the other three 
groups completed a fulfillment rating for each item. The -espense rate was 
approximately 57 percent for the mail questionnaire. The respondents were asked 
to select from 6 statements regarding the perceived importance of an item. 
These statements ranged fro- very high importance to no importance. 

The study selected as high importance items, all those which had been rated 
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either very I. jh or high. Fifty-seven of the items met this criteria and were 
identified as important elements for an effective CVE program; 
Two of the items Wiiicb supported the involvement of the administration in the 
operation of a CVE program were: 

■ - Ths administrative structure ^s flexible enough to serve the diversified 
needs of "individuals and distributive businesses. 
- The administration is responsive to serving the diversified heec s of indi- 
viduals and of distributive businesses. (p. 156) 

The issue of administrative responsibility for job placement was also 
raised by the McKinney et al . (1981) study of the factors related to the job 
placement of former' secondary education students in trciring related jobs. 
They found that high placement existed in schools where aiiuini strators ; 
prih'cipa i c , counselors and teachers viewed placement in employment related 
training as the primary purpose of the vocational education program and where 
principals in particular were committed to the placement of rormer vocati^.j 
ediJ^ tiofi students in jobs related to their training. High placement sites were 
?. ..• o cv *» acter i zed by the responsibility for job placement being spy rati among 
rt i ny school \ personnel ; 
Staff 

Indicators for a quality staff include the personal characteristics of the 
^ CVE coordinators as well as external conditions which may impact on the coordin- 
ators work role. Such conditions include: staff time, salaries, work ioad, 
expedience, accreditation and icj-jity. Of these conditions, the issue of salary 
{ Tor CVE coordinators was often overlooked. In the 288 items listed by Wubben^ 
p ( 1980) not one concerned salaries fcr CVE coordinators. 

h Benson (1982) stated that the principles of tenure and equal pay "make it 

/ virtually impossible for comprehensive high schools to keep their vocational 
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programs up to date arid competently staffed" (p. 55). Benson (1982) lists the 
ability to respond to local labor market conditions as an important attribute of 
a quality program. Since vocational salaries are highly effected by prevailing 
industry wages, the characteristics ot flexible salary scales has been included 
as a quality indicator despite the fact that.no empirical evidence exists to . 
support its value in a CVE program. 

McKinnty et al. (1981) also addressed the issue of equity as an indicator / 
of quality programs. He found that higher job placement existed at sites h ere 
the school staff had a racial balance similar to the community served. This 
criteria was stated as an "impression" derived from his integrating both the 
qualitative and quantitative data. Wubbena's (1980) study included several sta- 
tements pertaining to the issue of equity as it applied to program charac- 
teristics, but none of these were rated as high importance items. 

The iss'? of staff time as a quality indicator was raised by several ; 
stuaies. Both W,'bbena (1981) and Lloyd (1981) found that CVE coordinators need 
a considerable amount of extra time to devote to coordinating the overall acti- 
vites of the programs as well as to teach the related instructional class. 
Additionally, Wubbena (1981) and Lloyd (1981) strt J the importance of an 
appropriate coordinator to student ratio, tloyd (1981) found that 34 of 75 CVE 
coordinators rated student-teacher ratio as a primary reason for program suc- 
cess, while 37 out of 75 rated sufficient time for coordination as a primary 
reason for program success. In terms of program components needing improvement, 
Lloyd's (1981) study ranked teachers needing more coordinating time as the 
second most important item. 

The need for staff licensure, experience, and accreditation are also supported 
by the literature as essential elements of a quality CVE program. Wubbena's (1981) 
respondents considered the need for the CVE coordinator to have a valid teaching 
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certificate in distributive education as a high importance item; while Lloyd 
(1981) stated that a CVE coordinator shoald be certified in at least one vocat"' ~al 
area as defined by the United States Department of Education, dbhnsbn (198i) 
found that c Med; .qualified staff were one of the five categories of program 
qualifies which were significantly related to positive program outcomes. Qualified 
.staff included teachers who had more work experience, were more involved in 
professional organizations, and frequently attended inservice meetings. 
Faci 1 i ti es 

Several of the items in this 'category appear to be self-evident, in par- 
ticular the need for space, equipment and materials. Wubbena (1981) was very 
explicit on this issue and goes so far as to specify that a telephone be 
available in the CVE coordinator's office, as well as that a private area exist 
where the coordinator can counci 1 with students. Johnson (1981)- also cited the 
need for adequate facilities* and equipment in a quality program*--* 

The need for adequate transportation was addressed by both McKinney et al. 
(1981) and Walsh and Breglio (1976). McKinney et al. (1981) cited it as a signi- 
ficant barrier to high placement while Walsh and Breglio (1975) note that in 
several cities the inadequacy of public transportation coupled with the distance 
students were expected to drive to work stations limited the enrollment in CVE 
programs to those students who had the luxury or use of a private automobile. 
Instructi on And Training Materials 

Harris (1971) surveyed employers and teacher coordinators working with 
distributive-education programs in a five state area. Four hundred and ninety 
six (72%) .of the teacher coordinators and 544 (50.1%) of the employers 
'responded. One of the major objectives was to determine how important certain 
program procedures and operations were to the success of a cooperative pi an 
distributive education program. 
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Employers and coordinators were asked o rate the importance of nine 
selected items on a three point scale including! very important, important, and 
unimportant. All nine items were rated as very important or important. Of the 
9 items, "training materials for the student to study in school which are 
related to his training station experience" received the second highest ranking 
while "the need for the classroom instruction to be related to the work 
e ,j : r.ce of-" the student" received the highest rating. 

The importance of training related instruction is evident in the research 
literature. Benson (1982) cited comprehensiveness and depth of instruction as 
one of the three in factors of a quality program. McKinney et al . (1981) 
found that high placement existed where vocational education curriculum was 
oriented to the needs of local employers. Cushman et al . (1968) noted the need 
to coordinate " .he content and sequence of instruction with job requirements" 
(p. 36). Johnson (1981) found that a realistic, competency based curriculum was 
oru: of five program characteristics which had a significant impact on program 
outcomes. Wubbena (1981) reported the need to have related laboratory and 
project experiences. He also identified the need for the reading level of 
training materials to match the students reading level. Lloyd (198]) adds to 
this list the need for programs to develop students' basic educational^ ski i is as 
wen as vocational skills. Lloyd (1981) found that 64 percent of the W ning 
sponsors surveyed thought that the intellectual level of the students in their 
programs needed ' -provement. 

Affective work competences. A substantial amount of research indicated a need 
for instruction and training in developing affective work competencies (Beach 
and Kazan as, 1981) and that such competencies must address the beh* : idral , 
cognitive and attituciina' needs of students: (n practice, thi- trans 1 a, ;s si nto 
a ne-.>d to build employability skills, good work attitudes, appropriate work 
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behaviors, job seeking ski 1 1 s and jol* -oping skills in students. 

Mangum and Walsh (1980) in the! . of "do's arid Jon'ts" for work 

experience programs stated that programs must include the following skills: 
"working with others, controlling impulses, processing and 'nterpreting infor- 
mation, commun eating, problem solving, and work .,hj wilh an authority structure" 
(pq. 74). Lloyd (1981) noted that the CVE coordinators who ^ac.ght the related 
class placed a high priority on teaching con^'in i :at i oris s.^F c , attitudes and 
employer /employee relationships. This emphabi* seems well placed since training 
sponsors most frequently stated that the reason chsy • '-cted certain students 
was their "strong impressions about the students skills in the a^ea of human 
relations and the ability to commun i cat - K {' loyd. 1981, n. 12S). Lloyd (I98xj 
ranked attitudes, employer/employee relationships a- J communications skills as 
the three most important items of instruction for the CVE "elated class* 

Harris (?. , stated that instruction should je designed so that students 
develop trie j lU,; ng competencies at a higher lev^ 1 than presently possessed: 
"following directions, v. rking with pflo; anc, acceptance u,,u adherance to 
company policies and procedures" (p. 35). McK.inney et al . (1981) found that 
sites with high placement rates provided students wHh training in job recdinciss 
ski l is. 

Career exploration . Wubbena' s (1981) list of essential itc-i* for a t'E quality 
program included both instruction in human relation? ind occupational adjust- 
ment. Ci-ie of Cushman ' s et ai : (1968) high priori cy items for EVE programs was 
the a^'.ity of the program to help clarify a student's occupatioral choice. 
Mangum and Walsh u980) emphasized th ; need to allow youth to sample a variety 
of occupational areas and stated that special classes should be coordinated for 
the nu'-uose of enhancing career exploration. 
1 Vputh organizations . McKinney et al. (1981) noted that where there *S high 

77 




level ot placement., there is also a high level of student p rticlpatiun in youth 
organizations, in ,v nson's (i981) study; 50 state director, of vocational edu- 
cation identified student organizations as a dimension of pVOtj, an quality; 
however, he did not find a significant .ssociat ion between this criteria and 
program outcomes. Lloyd (1981) found that an active student vocational organiza- 
tion t-uis one of the pr'iii.ni y reasons for CVE program success and r -commended that 
an ideal CVE program include a strong vocational student 6rg>:ii zatien . 
:>[>••( i aj Pr ograms 

Several of the studies reported a need for quality activities to sup r 'rt 

the objective's of a CVE program. Mangum and Walsh (19BG) state 1, 

...the experience of the past seems to indicate that the er 
provision of jobs to youth, without program enrichment, .i . riot , 
effective in reducing school drop-outs, encouraging youth to return 
to school, V in improving the empl oyab i 1 i ty of youth, (p. '/?.) 

Fuiibwirid are some of the Mndings with'respect to two different Service? 

which have been ; ncluded in CVE programs. 

, ouu incj. Jonnson (19«i) found chat eounse I i rig service:, Mad a Significant 
impact on program -utcomes. This finding is supported by the work nf Eanaum and 
Walsh (19C0). Benson (1982), and Eloyd (1981) and Walsh and Breglio (1976). 
MariqiJUi and Walsh (1980) noted some negative findings on the value of counsel leg 
in a supported wo. < experience program, but concluded thai: it would he inad- 
visable to either curtail or expand existing o mseling services on rh<? basis o c 
the present research. 

The CVE coordinators in Lloyd's (1981) study listed the need for more coun- 
seling support as the dumber one prob'em which must be addressed in order to 
improve their CVE regrams. Benson ( 1982) noted that the role of the counselor 
is an especially important one in vocational programs, since he/she can advise 
s Events about skill and job requirements as well as help establish a strong 

link with local industry. Wish and Breglio ( 19/6) f oiiriu that 47 percent of the 
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students who hud discussed CVE with a counselor (62 percent of the participants 
indicated they had talked with a couloir-) thought that it ,vas eithor very 
helpful somewhat helpful; The option of EVE coordinators lowaM triLir. 
schools counseling services v?ats less favorable. 

Placement ser vi c es . Johnson (H81) f i-hd that placement v another of the five 
major characteristics which had a positive impact on p am outcomes. Man gum 
and Walsh ( 19L0) encouraged building in or at least it 1 rid outside placement 
services for students, and Lloyd (1981) recommended t .t Lath high school provid 
a placement service for students seeking full-time employment after graduation; 

On the negative side McKinney et al . (1981) found that more low placement 
sites nad at least one designated job placement officer than did high placement 
-, ites. At the high placement sites teache. s were included placement acti vi- 
ti?- ard the placement office provider the coordination for such activities. 
They concluded that although responsibility for job placement is important, it 
must be shared. 
Program r _ v - \V ' at s ' n s- 

An 1.:;pO"tant issue in education today is the concern for -valuation and 
f::cr'!.)dck . The literature clearly indicates the v .k! for students ih a CVE 
program to be evaluated and kep f informed of their program achievements through 
some type of formal training plan and tv • latiori instrument. 

Lloyd (1981) Found mat 96 percent of the teacher coord naturs in his study 
used training agreements while 82 careen t us-d trair ing plans. Harris ( 3 071 ) 
noted that both c jor J i nators ahd employers ranked t. e need for a training 
plan and training agre> ■■iiL i'. ^ Lner very important desirable for c pi r. 
Between tne two gr< ., re was a difference in terms of perceived import 

\ )( - t jif:, u . ; rs juG'Jed the i. r~a i ri i ny agreement as more important than a trainwiy 
pun, while eihpi- -irs rated the training n 1 an higher + nan the training agreement 
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Cishndri et. al. (1968) found that teacher coordinators ranked the 
"systematic evaluation of program progress and development soi'cated prin- 
cipally from student' and employers" (p; 35] as a high priority item. Students 
ranked program evaluation and feedback as an intermediate priority item but 
rated the need to discuss and resolve training halted problems in the classroom 
as a High priority item. Employers ranked the need for some kind of agreement 
establishing their responsibility as a high priority item. 

Despite the general belief that program evaluation and feedback is an 
important characteristic of a quality urogram, Lloyd ('931) noted thac "Teacher 
coordinators do not, in general, share with rtuuents : i; ■ written measurable 
objectives that they have developed for their CVF. programs" (p. 141). 
Summ ary 

A hot "i 1 of 31 school competent items were selected from the literature 
because they nave been identified as quality indicators (Table 1). They are 
a os I te of both inputs and processes - i;ch <ei>;; to be important in the 
overall operation of a successful C\ E program. These i o^is were dividr ; into 
seven subcategories rorsion 6'i inclusive of the components functionally admin- 
istered by the ;chooi itself. These include, student-, staff, administration, 
far. i I itihs, instruction and training materials, special programs and program 
p 1 ah hi rig and evaluation. 

Many 6 C Mie items listed are problematic in that they lack precise operd- 
. ,.„, .] def inition: For example, several studies indicated that the ;VE coor- 
dinator should H«,ie a rapport with youth. This item would seen to have at least 
Mi.|h face validity but very low construct validity. In few cov • is "rapp' : ' 1 
; .... a precise definition. This problem is ■ m i I- ^ral other indicators. 

.. n ,-,,! ..if., ,.his ,ist is the specification and range of "quality" 
that could p. e.-.m iy be fodhd in reveral of the i^ms. For ii, stance, lot pi ice- 
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ment servi es Have been found by at least one stt-dy to result in high placement- 
a major goal of most CVE program; However, this information does not K ^rmit 
determining whether or not any job placement service is better than hone or 
whether there is an important element of quality that must be present in the 
service; if this quality must exist, then it is important to determine precisely 
wh-.c distinguishes a quality placement service from a poor piacpneht service. 

At some point the assessment of quality car be so reductionists the' it is 
overly time consuming and costly. A very practical decision must be made 
regarding the trade off between feasibility and validity and between Pragmatism 
and idealism. It is easy to lose sight of the fact that the ultimo 1 v criteria is 
a student who is adequately prepared to enter the worK world and thai resources 
directed to this end may have to be allocated despite a 1 ack of conclusive evi- 
dence establishing a causal relationship between a program and its goals. This 
is particularly true when the i aw of diminishing returns suggests the cost of 
"conclusive evidence" is prohibitive; 

Job Gomp' ; 

An important dimension of a quality CVE program includes those charac- 
teristics related to the training site (see Table 2). lit^'ature specifies 
certain eharacterisu.es v,rvi ch a goed o-aihirig site must j cssess in order to be 
conducive to student growth and development. These elements reported ]y can 
either Lcj developed through appropriate coordination between the school and 
participating business or else through careful selection and careening of 
potenti^ train* rid sit*s. The indicators of a quality worksite na r/ e been 
categorized into f our areas: structural characteristics cf a ; *'ork site, job 
character i st i cs , interpersonal reiationsh : "in-i supervision 
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QUAbMY INDICATuR c FOR COOPERATIVE VOCATIONAL EDUCATION PROGRAM: 

JOB COMPONENTS 



Structural Characteristics of the Worksite 

1- Employers are in compliance with Federal, state and local laws. 
2. Job.sites. located in the private sector and with small companie 
pr ef errabl e. 

Job Characteristics 



Job content should be meaningful.- _ _ :_ 

4. Job should be challenging enough for the students ability level. 
5 Jobs should he related to students course of study. 

6* Duration o, work experience should be long enough for the students to per- 
form a variety of meaningful tasks. . . . 
7 Job activities should foster responsibility. 

8. Students should be cTfared the opportunity to explore the work site. 
9* StLdentb should indicate satisfaction v, : th their job, 
10. Jobs sho-ilr! provide an adequate amount of work to keep rtudents bury. 

Supervi sion 

done at the work site, 
to job duties and respon- 



it the vork site. 



Interpersonal Rel ationshi ps 

15 Appropriate adult role models at the work site. 

16. 'Vood social relationship between trainee and ^llow workers. 



ll a Adu't encoi-agement is offeree for * 

12. F Hy and thorough orientation of t ! 
si Dili ties by the employer. 

13. Adequate feedback related to job per 

14. Adequate supervision should be provide 
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^M'£t_ura_!_ character is tics of the Work Site ; 

Wubbena (1981) stated that employer compliance with Tedera;. ttaie and 
local iabor laws is a major indicator of a qUali rcgrem. His inclusion is sup- 
• ed by the often noted comment that CVE programs bhould pcy nri'Tmum wage and 
provide equitable remuneration for services rendered by student workers to an 
employer ("ui-hman et al . 1968). 

Several stuu'es reported that cdffiffijrcial sites were preferable to public 
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sites and that sm 1 i . • ; : : : preferable to large sites. Harris (1971) rioted, 
that 39 psre^n': e , . :e^chei ~tiri-J i r-a: '^s oelieved that firms employing be- 
t;-;eeh two and ten worr ; ' % s pi vi:\.d J e I. r ri i rig sites. This belief was 

reported more frequent!; i y \\ n coordinators than by urban coordinators. 
Mel; inney ec a I. ( I9t 1 ; fowusl V; it post-high school job placement rates were 
positively correlated wit', sites having more smaller industries and fewer large 
industries. He noted that tir, s could be due to the fact that, for one or more 
reasons, vocational education is not serving large industry. 

Cushman et al. (1958) found that teacher coordinators rated as a high 
priority item placing students in a "commercial out-of-school setting" as 
opposed to public service set + ings. Ball =md Gerould (1980) noted that in 
an analysis of 520 work sites, the very larg-a work sites (more than 25 youth 
employed* ^ended to be of Substantially lower quality than the sirial^r work 
sites (4 or less youth employed). In addition, the youth who wor<-?d in the pri- 
vate sector perceived more value in their work than youth working \t nonprofit 
agenc i es . 

dob Ehar aeter i st ' cs 

Owens and Owen (! q Si; conducted a :tudy of 1103 high school students 
enrolled in 18 Experienced Based Career Education (EBCE ) Proqrams in 16 states, 

is a program designed to integrate individual learning experience it* the 
school and in the community. Two of the key questions asked were: what 
happens at specific won; sites which enhance; or detract from student leaning 
and what job site characteristics do youth believe i.iake an excellent learning 
experience? Respondents were asked to indicate the degree to wh^ch various work 
site factors contributed to an excellent or poor learning expn; fence; The . 
results indicated that four out of che five items most associated with a good 
learning site had to do with specific characteristics of the job. These were 
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stated in order of importance: hands on activities, i.e., being able f o try out 
the .vork itself, being given challenging tasks to .^-form, having adult respon- 
sibility arid the content of learning was job spsc.fic. 

oOnrad and lied in (1981) found that students felt a good work site included 
Challenging tasks, the . portunity to do things themselves, instead of merely 
observing, a variety of ibrfci responsibility, the freedom to explore personal 
interests and ideas, and the feeling that one made some important decisions 
which could contribute to the companies goals. Silberman '1974) in his study of 
students in work education programs found that "meani ngf al ness of the work role" 
was one of the important contributing factors to job satisfaction. Man gum and 
Walsh (1980) emphasized that work training programs should net place students in 
jobs which required little supervision or made few demands on them. 

Employers in the Cushman ec al . (1368) study rated as high priority that 
students be involved in tasks which would ,'eve lop useful occupational skills 
and that the duration of the work experience be long enough to give students the 
opportunity to perform a variety of job tasks. Teacher coordinators rated 
students having a "re;;! job experience closely related to the students course of 
study" (p. 35) as a high prior 'iy item, while students] rated being prepared for 
a particular job as a n'oli :.r "• item. \ 

McKinney et al . (1981) i.l'^id that One character i st icLof-4 't : ' : placement 
^ites was that tlie vocational education programs placed students in jobs which 
were ri i ted ,o their training programs. In the Harris (1971) study, the need 
for classroom training to have a relationship to the student's job was r .inked 
number one in a list Of major .Important progre ;■ variables by teachers, coor- 
dinators aric employers. Walsh and Brey'no (lS7bj found that strong rela- 
tionships existed between the criteria of success end the high-y i ated 
integration if clc-sswerk and on * he- job traimno. ; 
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The Youth Incentive Entitlement Project was established under the Youth Act 
of 1976 (Public Law 95-93) to provide jobs in V communities for disadvantaged 
16 to 19 year olds, c;».!di ti onai on their returning to or remaining in school. 
Each youth enrolled in the project was provided with a part-time school jc- 
ful 1- time, summer job. Ball, Gerould and Burster; studied the project 

worksites in order to gain a greater u<,dersta< >j ' r tviiat constitutes a quality 
worksite. Ball et al . (19bOj selected a random sample of ?. 9 C work sites v.iiich 
had spor^ored youth ' .n the period of Septemhor , 1978 through November, 
1979. Nineteen expi u assessors collected data on a variety of job charac- 

terises through fie.j visits, where they recorded the perspective of the youth 
and .:L-..-k sponsors as well as their ow<~ perspectives. They analyzed statist^- 
colly Uose factors which memo the greatest contributions to the ratings 
rprcrded by the youth, their supervisors and the assessors. The following job 
characteristics were first identified through a literature review as ind v ? tors 
of a quality work site: 

1. Content of the job (skill level, perceived val \ie of the job; and 
intrinsic interest in :he tasks per r orm) . 

2. ExT.tn* to ^riich yo u \ h were kept t /. 

3. Youths' sw* eness of r.i^iJHr ds ah^ job duties. 

4. Yout'.s ' perception or the utility of th "*ork per-,, Tied (whecher o~ no; 
they felt they made a Coiitr; Juti oh to the sponsor's mission ai , : { if 
they f 3 1 1 appreciated for their work). 

5. Youths' overall satisfaction with their jobs. 

Ball et al . (1980) developed indexes for er-ch of these areas base^i on 
a composite of subchar acter i sties wh ' eh made up each of the major charac- 
teristics (listed above). An analysis of these characteristics indicated that 
the three major de f ^minents of wo |: vlte quality Abased on assessor ratings) 
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were (1) whether the youth we*- > : sy, (2) whether the yo th were S3tisf.it! 
with their job and (3) whether t"e joos had relatively high job content (p. 
71). Uetenriinerits based on you: ratings were related to a preference for joos 
which Mac a high skill content, ;5 : . ; 1 cjs and interring ?,. : ai ..ants, an ample 
amount of work to do and the same work standards fo, j^l'-. ■- for regular 
emp 1 byees . 

Supervision specifically refers ..o the manner in which the student is 
advised of his job responsibilities, 50b progress, and job performance. These 
processes include a need for feedback; however, feedback is merely one 
component of si per v i si on . 

Owens and uweri (1981) found that adult recognition for the student's work and 
Liihg given clear directions to follow were major reasons why ce. tain job sites 
resulted in excellent learning. In Conrad and Hedin's (1981) study, students 
indicated it was important that they received help when they needed it and that 
adults did not criticize them or their work. Man gum and Walsh (1980) stated 
that supervisors Should demand good work habits on t! ■ job without being overly 
strict.-. The study by Walsh and Bregl 10 (1 975) found that programs where the 
tra'niriy and superv.: ion was highly rat. d had higher success outcomes in the 
following categories: J 

1. Job satisfaction of students. 

2. Student school satisfaction. 

3. Overall student, rating of programs. 

4. The likelihood students would recommend program. 
Attitude towards pro; " . 

6: Attitudes toward jobs. ..p. 33} 

Siibfen.iah (19'5) found that feed' k was u"e of tne major factors wHich 
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contributed to the job satisfaction of work experience students in his study: 
He note- that CVE coordinators should assign students to sites either where such 
feedback is available or where. the coordinator can assist the employer in devel- 
oping the skills to provide such feedback. An emphasis on feedback is also 
shown in thr Cushman et al . (1968) study where the teacher coordinators rated 
as a high priority i:em the 11 systematic evaluation of program progress and 
development solicited pri nci pal ly from students and employers" (p. 35) - In the 
Cushman et al ; (1968) research, • al 1 three respondent groups also rated as a high 
priority item the need for a thorough orientation of the students to their job 
duties either very early or even be -^e beginning work experience. 

Ball et al . (1980) listed ariiorr the 14 major determinants of a quality job 
si f e Five items which support the „ed for good supervision, these were: (1) 
youth understand duties, (2) yo ; are informed of attendance and performance 
standards, (3) supervisor and yc i interact frequently, (4) participant to 
supervisory ratio is less than ■ r;e to five and (5) the assessor judges- "^he 
quality of youth/supervise^ interaction as average or above average. The 
~ c essor's judgment was guided By a field instrument which encompassed a number 
:' both general and specific fa.cr.ors. Plated to super vi sory/ youth 'interaction 
there were 13 characteristics specified which the assessors took into account in 
their determinations. These included: 

•1. Supervisor h>d experience doing tasks required of youth. 

2. Silver vi sor had experience teaching tasks required of youth. 

3. Supervisor staff works in close proximity to youth. 

4. Supervisor speaks frequently with youth (general statement). 

5. Supervisor speaks with youth about tasks (general statement). 

6. Supervisor speaks with youth informally (general statement ) . 

7. Supervisor states he speaks with youth about tasks. 
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3. Supervisor states he speaks with' youth informally: 

9. uth state they speak with supervisor about task.-; 

10. Youth state they speak with supervisor informally: 

U; Staff usually available to answer youth's questions. 

12. Youth feel supervisor helps 1 them do better job; 

13; Worksite assessor judges quality of youth-supervisor interaction 
to be above average. (Ball et al . , 1980, p. 38) 
Owens and Owen (1981) noted that students with higher grade point averages i 
and Hispanic students were -sr. ' : --iv to value being closely supervised than 
were students with yr;e e point averages. This illustrates a potent" al 

difference between some of the attitudes held by youth. While some students may 
want to be free to explore, taKe i rs. pons i bi 1 i ty and do things on their own, 
others may place less emphasis on these characteristics and more on a need for 
closer supervision: Individual differences emphasize the need for careful coor- 
dination on the part of the work supervisor. 
Interpersonal Relationships 

An important element of an individual ' s job success stems from their ability 
to get along with their co-workers. This point was eniphas - ! in the literature 
vhirh explored youth i ati i aides and war': adjustment skills. 

Owens ahd Owen (1981) found that there was a significant difference between 
excellent and poor learning sites based on the relationship students had with 
adults at the worksites. Students at the e: ..e lent sites rated their rela- 
tionship with adults as significantly; * than students at the poor 
sites. S^Mrrman (1975) found that one < ' '^s contributing most to job 
satisfaction was the | avail ability of adult role mouels. Ke suggested that coor- 
dinators should 1 '.ok for site- with "assessable adult models". 

! 

In the Harris (1971) study, one of the activities rated bv ">th coordinators 
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and employers as highly important was the assignment of a specific individual 
Jo serve as an on-site training sponsor for the student. Conrad and Hedin 
viy8i) reported that the development of a personal relationship with someone 
at the work site ranked fifth among the 14 most important characteristics 
contributing to student growth. 

Another additional interpersonal characteristic which was noted in the 
literature as being an indicator of a quality program wa^ the students' ability 
to develop good social relationship. .;i th fellow workers within a congenial work 
environment. This is not necessarily the same as developing good work rela- 
tionships with co-workers and supervisors. \ i 1 bermans (1975; cited group 
atmosphere as an' important variable contributing *;o job satisfaction. He 
described this as "th- social climate of the work team, trie availability of 
supportive stimulation, and sense o^ family oh the job" (p. 266).. 
Summary 

The job components category of quality indicators included 16 items which 
were divided into A subcategories. These included: structural charac • -ri sti cs 
of the work site, job characteristics, supervision and interpersonal relation- 
ships. T 1 e brief exami nation of some of the research which looked at causal 
re 1 '.ships between these items and program outputs was far from concl usi ve. 

One major weakness of most of these indicators is that the judgment of 
their ''quality'' stems from self-ratings, oucu measures are only one potential 
method of establishing the validity of a specific treatment. Before it is 
possible to determine with more confidence that these items are in fact quality 
indicators, an effort must be made to link these measures with specified program 
outcomes. 
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Coordination Activities 

A major difference between EVE programs and unstructured work study 
programs derives from the effort in CVE programs to synchronize the students' 
employment experience with their educational activities. This requires the 
addition of a number of linkage mechanisms to facilitate the interdependences 
between the school and the employment site. Since the CVE coordinator has the 
primary responsibility in this system, his/her role is considered the vital ele- 
ment in insuring that the school and work site are in harmony. 

Nine specific items were identified which summarize the most critical indi- 
cators for coordinating CVE p-ograms (see Table 3). Coordination indicators are 
divided into four categories; employer responsibilities, coordinator's respon- 
sibilities, joint evaluation, and third party input. Each of these activities 
are needed to insure a functional relationship between the educational and 
employment components of a CVE program. 
EmgjjDyer_ RGS pons i bi 1 i 1 1 es 

Both coordinators and employers in the Cushman et al . (1968) study rated 
allowing the employer the right to interview and select the applicant of his/her 
choice as a high priority item. In addition, students rated as a high priority 
item "obtaining placement for work experience through the normal hiring process 
of application and interview" (p. 36). It seems logical that employers would 
have a better attitude towards those programs which respect their right to 
select employees. 
Coo rdinator's Responsibilities 

The coordinator's main responsibilities (outside of the educational 

institution) can be summarized as: making an adequate number of supervisory 

visits to students and employers; conjointly planning and managing the 

employment experience with the site sponsor; being professionally involved in 
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the field; arid providing the employer "with appropriate input for conjoint 
evaluations of the student. 



Table 3 



QUALITY INDICATORS FOR A COOPERATIVE VOCATIONAL EDUCATION PROGRAM: 

COORDINATION ACTIVITIES 



Employer Responsibilities 

1. The e-nployer has the prerogative to interview and select the student applica- 
tion of his choice. 

Coordinator \s Responsi bi 1 i ties 

2. Teacher coordinator is professionally involved in the field. 

3. Coordinator has a training agreement to insure training sponsors are aware 
of their responsibility and performing appropriately. 

4. Follow-up results are obtained for students completing or leaving the program. 

Joint Evaluations 

5. The employer and coordinator are jointly involved in the evaluation of the 
student. 

6. Teacher coordinator makes an adequate number of visits to employers of 
cooper at i ve students ; 

Third Party Input 

7. An advisory council exists to ensure community and employer input and 
support the cooperative program and for program development. 

8; Program should obtain local union support. _____ 
9. Parents are contacted to help determine student needs when appropriate. 



Making an adequate number of supervisory visits to employers has consis- 
tently been noted as one of the major indicators of a quality program, [ 6 ashman 
et al . , 1968; Harris, 1971; Lloyd, 1981; and Wubbena, 1981). The coordinator' s 
responsibility for developing a training agreement and a training plan with site 
sponsors has also been well established, (Cushman et al., 1968, Harris. 1971; 
and Wubbena, 1981). The purpose of the training agreement is to insure that the 
student and employer are aware of their mutual responsibilities concerning pay, 
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length bt M-ivicc, Hours of work, etc: A training plan details what learning is 
to take place and what competency levels mast be reached by the student trainee: 
Despite concurrence on the value and heed for adequate training plans and 
agreements there is some evidence that they are not always used. 

Iil the Walsh and Breglio ( 1976) study of urban CVE programs, 73 percent of 
the participating employers interviewed reported that they had not signed any 
formal agreements with the schools, while less than one-half of the coordinators 
reported developing any formal training agreements. Walsh and Breglio (1976) 
noted that "in most cases, no training plan or contract stipulations were 
written for employers to follow" (p. 18). They further reported that since most 
employers participated in grading students "without specific training plans... 
employer ratings of student performance ... were only marginal 1y related to 
educational objectives" (p. 13). 

The most ambiguous of a coordinator's responsibilities concerns his or her 
professional competence as well as dedication and commitment to the profession 
and students. Johnson (1981) found greater positive outcomes in programs where 
teachers were more involved in professional organizations, had more work exper- 
ience and attended more i riser vice workshops to upgrade their job knowledge. 
Benson (1981) cited as a second major attribute of a quality program a sense of 
closeness between school and industry. This meant that teachers and employers 
had frequent contact with each other and often visited the others environment as 
well as lending equipment and skills When needed. Lloyd (1981) stated that in a 
quality CVE program, the teacher coordinator frequently attends inservice work- 
shops, contacts employers about job openings for students and works with the stu- 
dents' vocational organizations. Two of the criteria which RcKinney et al . (1981) 
found associated with high placement were a high level of staff enthusasium for 
job placement and a need for teachers to have regular contact With employers. 
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Such duii\icU;rist i cs of professionalism as involvement in the field; contact 
with local industry, and on-going training may requ're a considerable. 1 amount of 
a coordinator's time. Since these are not often the Kind of acti:'ties which 
a coordinator is directly contracted to complete, it may be necessary to provide 
additional motivation for coordinators. 

One important coordinator responsibility is tiie need to obtain follow-up 
results for students completing or leaving the program. Lloyd (1981) recom- 
mended that a quality CVE program conduct f o ! low-ups of former students, while 
liubbena (1981) stated that staff members should follow-up all student? either 
completing the program or leaving the program with a marketable skill. Finally, 
Cushman's et al: (1968) study found that although teachers did not rate ~er iodic 
follow-up studies of students as a high priority item, they did rate it as an 
item of intermediate importance. 
Jo i nt Eva! uat i ons 

The two major tasks that must be addressed by joint evaluations of CVE 
programs are: the need for the training sponsor to be involved in the students 
on-the-job evaluation and the need for students, coordinators, and sporscrs to 
jointly participate in the development of a training plan (Wubbena, 1981). 

In the Cushman et al. (1968) study, teachers rated as a high priority item 
the solicitation of information from employers and students that could be used 
to evaluate student progress. Students rated the need for employer input into 
evaluations as an intermediate priority item, while student evaluations were not 
rated as a priority item by employers. This last finding may reflect the loss 
of time which an employer feels is involved with evaluations as well as the 
general distaste many employers have for employee evaluations. 
Th ird Person Input 

Third person input concerns the need for CVE programs to coordinate 
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their at: t j v i L i es with other parties who may have ah impact on the CVE Program: 
This includes, not only employers in the local area, but also the community, 
local i ibor organizations and the families of students. More effort has been 
(ii>viU>d to examining the role of the advisory council in CVE Programs than the 
ne'Hl for labor '-upport or direct involvement of parents. Research info the role 
Ot tii advisory council is almost unanimous in qualifying it as one of the major 
in!' afors of a quality program. 

iii liu'siiinai- 1 s et al. (1968) study both teachers and students rated an advi- 
sory v.ouhcii and employer input into the planning process as a high priority 
item; bur pr I s i ng 1 y, employers rated the functions of an advisory council as a 
high priority" item but rated the establishment of such a council low. Cushriiah 
et al; (1968) noted this inconsistency and stated that "it is probably aitribu- 
tahle to employers concurrences with specific advisory functions, to which they 
did not avert in evaluating the advisory boards establishment" p. 39. 

Johnson (1981) found that 30 percent of the state directors listing 
quality indicators agreed that advisory councils were essential. In his final 
ina 'y,i Johnson (1981) noted that advisory councils were one of the five major 
determinants of positive program outcomes. Programs liSiilj councils generally 
obtained information which led to higher levels of student satisfaction and 

higher placement rates. 

In the Harris (1971) study, the advisory council was ranked fifth by 
employers in the list of n ne most important activities: Wubbena (1981) did not 
include an advisory council as one of the most important characteristics of a 
quality CVE program, but did if elude the need to "involve or inform employers, 
parents, faculty, students, counselors and civic groups" (p. 153) as an impor- 
tant characteristic of a quality CVE program. The majority of Wubbena 1 S (1981) 
respondents rated the functions of an advisory council as at least having some 
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importance . 

Despite the amoral support: for an advisory colinci 1 , Lloyd ( 1981 J round 
that when CVE coordinators were asked to compare their program to a list of 
ch ractoristics which least contributed to programs success, they listed the 
fo! ;n W i ; ij: active ;'se of an advisory council, intellectual ability of student 
( - ; nd :-norci\itiori oi tin i oris- Twenty percent of the teachers replied that the 
advL-c; ; y council had rarity or iiev^r contributed to pr^ram success while 39 
percen of the teachers indicated that it had frequently contributed to their 
programs success . 

One possible explanation of Lloyd's (1981) findings is that the involvement 
' the advisory council determines whether or not it is seen as a positive fac- 
tor. In Lloyd's (1981) study, 28 percent ot the coordinators reported that 
their advisory council met only once a year while 30 percent reported that their 
council met three or more times each year: 

In the area of union supoort for CVE Programs, the findings are mixed. The 
coordinators in Lloyd's (1961) study did not find unions to be very helpful to 
their program. Fully 90 percent of the coordinators indicated that the union 
gave "no visible assistance". Only one teacher indicated that it had been help- 
rul while six teachers (8 percent) indicated that the union had actually ham- 
pered their program. Walsh and Breglio (1976) did not find union opposition 
to be a major constraint, however most of the work stations for their students 
were in non-union areas. They indicated that any adverse influence by unions 
has yet to be tested. 

Frankel (1973) advocated that coordinators actively encourage unions to 
participate in CVE. He noted that administrators and teachers are often 
reluctant to pursue jobs or training spots in union organized businesses. He 
suggested that by more actively involving unions in an advisory role, employer 
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reluctance to employing non-union trainees can be overcome. Frarikel ( 1973) 
found that programs with active union involvement directly benefited their 
students in terms of job placement and also because with union participation it 
was more likely that student training would count towards subsequent appren- 
ticeship requirements. 

The final third party indicator is the need for parental cooperation. 
Tins characteristic should be distinguished from the need to include parent 
representation" on an advisory committee. Parent cooperation refers to the need 
for counselors and coordinators to directly solicit the input of parents 
regarding needs or problems affecting their children. The evidence to support 
this indicator is slight since little research has been done on the role of 
parents in the CVH process. 

Students in the Conrad and Hedin (1981) study ranked the discussion of 
school and work experiences with family and friends as seventh in importance out 
of a list of 13 items which they felt most contributed to their growth. Unfor- 
tunately, Conrad and Hedin (1981) did not distinguish between the effects of 
parental "influence and peer influence. Wubbena (1981) listed as a high priority 
item for a CVf program that each member of the school staff "contact 
parents ... to help determine students' needs" (p. 167). 
Summ ary 

The coordination activities category of quality indicators included nine 
items. These items were divided into four subeategor i es : employer responsi- 
bilities, coordinators responsibilities, joint evaluations and third party 
input. A close analysis of these items raises the same questions concerning 
definitions, specificity and val idity which have already been raised regarding 
the indicators in the school and job components. Again, since most of the items 
were derived from self reports or ratings, there is little evidence linking 
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their impact to program outcomes. 

Few studies have attempted to go beyond the use of expert opinion, and 
those that have usually attempt to establish a relationship between one or more 
indicators and some type of economic outcome (for example a relationship between 
coord inat or/ student rapport and placement rates). While such efforts have 
considerable merit, there is a need for additional efforts to link processes and 
outcomes using a broader and more comprehensive set of program outcomes. 

Most of the rationale for including an indicator presently derives from a 
concensus that the item is a quality indicator. Such a concensus should not 
be slighted, however, neither should it be confused with conclusive evidence. 
The responsibility remains for proponents of CVE programs to establish a posi- 
tion link between their proposec list of quality indicators and targeted program 
outcomes . 

Conclusion 

The term inputs has been used to refer to a bread category of resource 
endowments which include land, labor, time, capital and equipment. Educators 
Have considered the impact of these elements on their programs, and limited 
funds make their consideration critical. In the day to day operation of our 
educational programs, it is vital that we not only have quality teachers, 
quality instructional materials, and quality facilities, but that there be 
enough of these resources for efficient and economically rational program 
operation. 

The literature indicates that although there is usually a concensus on 
which resource endowments are a critical part of a quality program, seldom was' 
any formal effort made to gauge either the quality of these inputs or their suf- 
ficiency. Ideally, society will provide resources for all of the students who 
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could and want to benefit from the advantages of an efficiently operated CVd 
program. The necessity to ensure the quality of our inputs must be matched by 



an effort to ensure that sufficient amounts of these inptts are available.. 

A substantial portion of the research reviewed in this section was con- 
cerned with the processes that are involved in a quality program. Processes 
organize or transform inputs in order that a set of preestabl i shed goals :an t>- 
attained. There were three categories to which processes were assigned: 
school, job and coordination activities. 

In the school category, effective instruction, administrative assistance, 
supportive programs and planned program evaluations were all identified as major 
elements of a quality program. Important processes in the job category 
included: adequate supervision, meaningful job respons i bi 1 1 t i es - appropriate 
coordination With school activities and high camber working relationships: 
rinally, quality coordination processes included: a clear acceptance and speci- 
fication of responsibilities on the part of the instructor-coordinator and work 
supervisor for student workers, planned joint evaluations and the solicitation 
of relevant feedback from interested stakeholders and families. 

If all of the above activities are undertaken, will this be sufficient to 
insure that program goals are accomplished? Although educational programs are' 
often systematic and directed to some end, they are often inadequately monitored 
to determine if their ends have been met. Often the only indication comes far 
too late in the process for any corrective measures to be taken. The concern 
witn this problem has been a recurring theme throughout this paper and while 
there are still rid satisfactory answers, one possible solution will be outlined 
in the following section. 
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CHAPTERV. 
IMPblGATIONS 

The task for vocational educaljrs is to create effective programs for 
addressing the needs of youth and society. CVE programs appear to be one impor- 
tant element in such a strategy. However, EVE programs cannot meet the needs of 
all youth; they are neither a care-all for youth unemployment nor a complete 
solution to such educational problems as high school dropouts. It is important 
t-j koep in rtii rid ^ that as Ski ar, Mollis, Byrne & Tollett (1983) caution, no single 
program can meet the needs of all youth. Quality CVE programs are, however, a 
viable strategy for accommodating the educational and training needs of a signi- 
ficant portion of the high school youth population. Actual on-the-job training 
and supervision, periodir performance evaluations by both teacher-coordinator 
and a work supervisor, coupled with closely related classroom instruction 
demons ;rate to many people that CVE is a valid educational methodology. 

Nevertheless* the continuing tension created by different views of the pur- 
poses of education demands evidence of the value of educational programs and 
methods. CVE prig rams as a method of instruction must also meet the public's 
demand for accountability. Ultimately, professional educators involved in CVE 
programs must strive not only to demonstrate the effectiveness of CVE programs 
but also to develop methods to improve CVE education. 

An information system is an integral part of the effort to justify and 
"improve CVE programs. Emphasizing the need for decision information in planned 
evaluation efforts, riecessi uates lh expansion of the original CVE model used to 
structure this review. It adds a formative evaluation component which can pro- 
vide ongoing program information and a summat i ve evaluation component for 
assessing the overall strengths of the program relative to its goals and in com- 
parison with those of other school systems. Figure 5 aepicts the revised CVE 
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Figure 5, CVt Model with Evaluation Components. 



model with these additional components. These components are a necessary part 
of a system for the successful operation of a quality CVE program; 

This review has characterized the major elements of a CVE program in terms 
of inputs and processes and outcomes. Outcomes were looked at through a series 
of questions which addressed several issues instrumental in establishing goals 
for CVE programs. These issues pointed out the need to develop a consensus on 
the goals that a school has for its program, ideally, consensus is a necessary 
antecedent for a CVE program and will involve input from the individual stake- 
holders in the system: teachers, administrators, employers, parents, students 
and concerned taxpayers. Realistically the goal establishment process is 
completed in the political arena and compromise may be the best word for the 
results. The challenge is to Provide the participants with appropriate alter- 
nati ves . 

Inputs and processes were divided into three categories: school, job and 
linkage. In each category a number of elements were identified which are 
necessary for an effective program. These elements were labeled as quality 
indicators. A system for monitoring' the effectiveness of these indicators must 
be discriminating enough to select items which will lead to desired program 
goal s . 

Once a set of "indicators are selected, the next step will be to periodi- 
cally monitor (formative evaluation) whether or not the program is 1 unctioning 
as intended. This can be accompli shed thru either formal or informal proce- 
dures. Formal procedures should include student and employer surveys, while 
informal procedures could include group discussion or meetings with a student 
committee established for this purpose. 

Program outcomes measures must consist of items which will enable program 
planners to determine if program goals are being met. Since the goals are the 
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heart of the system, unless effective outcome measures are selected it will be 
impossible to know whether or not the system is functioning appropriately. 

Each of these three major elements, goals, inputs and processes and out- 
come measures can be linked through the use of a procedure that integrates the 
formative and summative evaluation process into a comprehensive system for the 
overall guidance and direction of the CV : " program. In order to accomplish 
this, the process must be more than merely a vehicle for data retrieval. 
This new system added to our original model may be defined as an instructor- 
coordinator information system. Figure 6 is a model of such a system for the 
analysis and improvement of CVF programs. The model suggests that the key 
elements of the system are the processes of goal setting, data collection and 
summation, analysis of data relative to goals and guidelines, and information 
dissemination. In Figure 5, these elements Were simply labeled as summative 
evaluation and formative evaluation. 

This information system will encompass the following activities: 

i: Clarification of key CVE program outcomes valued by local leaders. 

2. Establishment of local standards for CVE program outcomes. 

3. preparation of instruments to obtain outcome and input indicator data. 

4. Establishment of a system for analysis and reporting of information. 

5. Analysis of the CVE programs: comparison of results and outcome 
standards. : 

6. Specification of possible ranges. 

7. Dissemination of local pre gram information. 
Clarification of the Key CVE Program Outcomes Valued by Ig^J-T^aders 

The specification of program evaluation criteria for local schools has fre- 
quently been only the acceptance of criteria set forth or perceived to be set 
forth in Federal and State funding legislation and regulations. While the pro- 
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cess suggested for the establishment of local CVE program goals does not ignore 
these mandates, it also recognizes the importance of considering the effect of 
local "support groups" and their expectations on CVE programs. Since these 
groups provide the majority c ? the resources and have the most direct control of 
program operation, their interest must be a factor in analysis. 

A potential strategy for clarification of "key outcomes" is a modified 
del phi procedure. A list of commonly accepted outcomes for CVE programs 
can be prepared and submitted to individuals who "have immediate involvement in 
the CVE program. This includes: advisory committee members, instructor- 
coordinators, administrators, faculty, school board "members, and other locally 
selected individuals judged to represent key community interests. Each individ- 
ual should be asked to review the list of outcomes to add items which they 
believe ire needed and to rate all items on an importance scale. This informa- 
tion can be summarized to provide raters with the mean ratings by group. They 
can then be instructed to eliminate a specific number of "least important" 
items. Individuals involved in the rating process should be given the oppor- 
tunity to present arguments for the retention of items which they believe are 
key outcome measures of program quality but would be in their list for elimina- 
tion. A committee of five or six persons should be selected by the advisory 
committee to finalize the key outcome list. 
Estaol ishment of Local Standards of CVE Prog ram Outcom es- 

Eaeh community will need to select appropriate measures of outcome and 
standards for program quality. The effort in this step may be simply an edu- 
cated guess because validated external standards are not available or may not be 
appropriate for the local situation: Once these targets are set, they will 
serve as benchmarks for decision making and as the basis for communicating 
program status to the public: 
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The purpose of these targets is to assist a local school in developing stan- 



dards and to encourage critical analysis of these standards. While satisfaction 
of local program goals is a primary concern, one must look at the performance of 
others to clarify the reasonableness of local program outcomes. Consequently, a 
coordinated effort should be undertaken to consolidate data from a variety of 
schools to provide external standards for comparison -- external validity indi- 
cators. These standards might be a range of values for each outcome. 
Preparation of Instruments to Obtain Outcome and Input Indicator Data 

The instructor-coordinator information system proposed is concerned not only 
with data which allows determination of program status but also with information 
which allows assessment of key process and resource inputs. The challenge in 
the selection of input items is to identify only the critical processes, 
resources, etc., that must be satisfied if significant loses in program quality 
are to be avoided. 

It will be necessary to develop a set of instruments to gather the data for 
outcomes and process items from CVE students, instructor- coordinator, on-the-job 
supervisors, and building or school administrators. In addition, the instructor- 
coordinator's CVE program management forms, such as the training plan, student 
evaluation reports and student weekly reports can yield much of the data needed. 

The outcome measures will require a follow-up of CVE students, probably a 
short term follow-up approximately six months after graduation using a standard- . 
i zed follow-up procedure such as the Minnesota Secondary School Follow-Up 
Survey. This procedure can be modified for local CVE completers. 
E stablishment of a System for Analysis and Reporting of Informat ion 

To aid in the interpretation and use of data, the analysis can be presented 
as a set of tables displaying the summary data for comparison with established 
local outcome standards and eventually a "target range standard. 
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Analysis of CVE Programs: Comparison of Results and Outcome Standards 

Since a signficant amount of data will be accummulated by the end of the 
school year and a follow-up phase is also envisioned, analysis comparisons may 
be undertaken at two different times. The process involved will be similar each 
time. 

The information gathered to measure attainment of outcomes will be compared 
to established standards by the local advisory committee who will decide if 
program- goal s have been met. 
Speci fication of Possible Changes 

If an outcome goal is not met, possible changes are considered which my 
include: (1) revising standards, (2) revising data items, and/or (3) revising 
program processes or resource inputs: These changes Will be based upon the 
analysis and interpretation of data gathered. The instructor-coordinate, 
information system suggested here assumes that the process/resource inputs are 
causally related to the outcome measures, e.g., the quality of work site super- 
vision is expected to affect the growth and development, of a student-learner. 
The purpose of studying these relationships is to isolate critical process and 
resource inputs and to establish minimum threshold levels for these inputs. 
Dissemination of Local Program Information 

Greater public awareness of local CVE program outcomes is assumed to be a 
need. Consequently, dissemination of the evidence associated with program out- 
come goals is a critical objective. Not only the advisory group, but the key 
support groups and general public should be provided appropriate data on program 

successes as it is available. 

CVE instructor-coordinators need to continue- to demonstrate to the public 
that the program is more than "students leaving school to work part-time." 
The proposed instructor-coordinator information system is not a new eoneept. 
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It is a renewed call for a systematic treatment of planning arid evaluation 
information plus a call for an eff.art to establish local data bases that allow 
acetirmil ation of information across schools. This eff ort will lead to trie 
es tab! i shment of valid standards for outcomes, processes, resource inputs arid 
an analysis of the relationships between these elements. 
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